Artificial intelligence can assist software delivery and be used to automate software testing and optimize project work. Dimitar Panayotov uses ChatGPT to generate test data, create email templates, and produce explanations based on test results. This saves him time that he can invest to become more productive.
Dimitar Panayotov spoke about how he uses ChatGPT in testing at QA Challenge Accepted 2023.
Artificial intelligence (AI) is a collection of algorithms and neural networks that have been fed enough data to assist IT professionals, Panayotov said. That makes it a very advanced helper tool for assisting in the daily life of a quality assurance engineer, from information dumps on specific subjects, test documentation, and case writing, to workflow guessing patterns, he added.
ChatGPT is great at generating test data; this is its biggest strength, Panayotov said. It can also create test strategies based on requirements and based on best world practices, he added. It can write test cases, but it needs additional support because it does not train enough.
ChatGPT can be used for generating email templates and explanations based on test results, as Panayotov explained:
It can give a better human explanation for the test results that are usually only graphs and numbers for non-technical people. Also, it can create tables based on input.
Because the AI model is still in development, there are some bugs and glitches that may occur while using it, as Panayotov explained:
Overfitting with data (the information imputed by the developers of ChatGPT may be incorrect or derived from a common or faulty source) will give incorrect answers or missing values when asked.
Hallucination is another major problem - basically, the model tries to compile an answer based on the string that the user inputs, but because it has some limitations and rules that it has to follow, the answer may come as a complete fiction.
ChatGPT is a more powerful software tool than for example Google or Reddit, or even the all-knowing Stackoverflow, Panayotov said. Use it as a regular tool, know the product limitations, and don’t misuse it for maleficent purposes, he suggested:
It will save you so much time that can be invested to become more productive, for instance by learning new software languages and tools or expanding your working capabilities.
InfoQ interviewed Dimitar Panayotov about how he uses ChatGPT in his daily work.
InfoQ: How do you use ChatGPT for automated testing?
Dimitar Panayotov: I use ChatGPT for many operations that shorten the creation and execution time of automation projects and CI/CD pipelines. Here is a list of functions I use the model for:
- Preparing test plans and test scenarios- the model is very useful when creating new test scenarios based on web, mobile, or Cucumber test cases.
- Creating scripts in different languages- the general knowledge of the model for each programming language is just mind-blowing.
- Designing test cases- it can create specific design patterns for specific business logic based on universal models.
- Creating email templates- generating test data is one of the biggest strengths of the model itself.
InfoQ: What about using ChatGPT to optimize software project work?
Panayotov: Here are some examples of what the model can do for you to help you improve the speed of execution, optimize resource usage, and structure of your project:
- Format data and write documentation- documentation writing for a project is the most boring thing in the life of an IT specialist. The model can generate it, but it only does it if you give it the structure of your project.
- Implement CI/CD pipelines almost from scratch- with a simple request and with specification, the model can generate scripts for execution by almost all software and products that are in the CI/CD family.
- Create specific classes- if asked, the model can generate specific classes for your project based on its structure.
- Code reviews and error handling- the model can do basic code reviews of your code and even find vulnerabilities, but you need to know that your data can be used in future model answer generations.
We have to be sure about whether we want the model to have access to our project and give private information. ChatGPT is an open-source product; it wouldn’t be wise to input any client or personal sensitive information.
InfoQ: What’s your advice for testing with ChatGPT?
Panayotov: Like I always say: work smart, not hard. All manual and data entering and data generating can finally be skipped, as the ChatGPT model is made for that sole purpose, and more. Create as much dummy data and basic project structures as you need. Just don’t add personal or client data, because the use of it can be compromised in future versions or by data mining.