Supercharging Your Test Automation Code With AI Assistance In Your IDE

Hi everyone. I want to present some great ways to use AI within your IDE. 

These days, conversational AI like ChatGPT is integrated directly into the most popular IDEs like Visual Studio Code, Visual Code, and the JetBrains family of IDEs.

First I'll review the many use cases where I have found AI helpful in writing test automation code. Then I'll suggest a few AI tools you might want to try.  

Getting Quick Help With Syntax: AI Coding Assistants

An AI coding assistant can help you to write code faster and with less effort. The Continue and GitHub Copilot extensions provide this service. 

A coding assistant offers coding suggestions as you type. Sometimes you'll get a suggestion on how to complete the current line, and sometimes the assistant will suggest a whole new block of code. You can accept all or part, of a suggestion, or you can ignore the suggestion and keep typing. An AI coding assistant helps you write code faster because you don’t spend time writing boilerplate or simple code.

Example

GitHub Copilot code suggestion for a binary search function

Partnering With AI For Pair Programming

You can use an AI assistant as your pair programming partner. If you are a test automation programmer you have likely had to write some boilerplate initial code, create initial classes, or sketch out initial architecture. These tasks can take not hours but days, even for skilled programmers. 

In these cases, conversational AI can be really helpful. You can simply copy and paste your own description of the task and ask AI to start writing some initial code. AI does the task in seconds, while on your own the task could have taken you a week or more. 

This approach has downsides like outdated code or hallucination in a suggestion. About these downsides we'll say more later. But overall this is a great way to overcome the “fear of a blank page” and start coding.

Example

Request to AI: 

Create class with API endpoints for creating, retrieving, updating, and deleting user info.

AI response:

Boilerplate code suggested by AI assistant

Using AI As A Search Engine

If you've used Google or Stack Overflow search for programming answers before, you'll love being able to ask AI similar questions right within your IDE. The other day, I realized that for the last year I've been using AI search from within my IDE instead of Stack Overflow. 

However, there is a knack to it. You need to remember that AI systems can hallucinate and provide you with outdated information.This has to do with the architecture of neural networks that AI relies on. The training data that AI uses can be outdated. Therefore, you must verify the information provided by AI systems and check all libraries or suggestions AI recommends to you.

Example

Request (paraphrased for clarity): 

“I'm using FlaskRedis to work with Redis. How do I retrieve the Redis task if all I have is the switch?”

AI response:

AI code suggestion of how to find all keys in Redis via the FlaskRedis client if I know only part of the key.

I used the first code block that was suggested but it didn't work. It suggested a keys method, which may have existed in the past, but no longer does. This is a good example of how AI can hallucinate.

I then made another request to AI with a self check of the previous suggestion. This time, AI returned a working suggestion using the valid scan_iter method. Additionally, the assistant notified me about possible performance issues if I used that method. 
 

Example of how AI can make a self check

Diagnosing And Resolving Errors Using AI

AI can explain stack traces and errors. This is great when you're just learning how to use a new library or framework, or even a language that's new to you. Usually in these cases you can spend a lot of time searching for the answer. 

But now you can simply paste an error message into an AI chat and get a powerful, useful explanation of your error. Moreover, an AI assistant can provide you with a possible solution to mitigate the error or avoid it in the future.

Example

Request to AI: 

Fix an error

Code

client = FlaskRedis()
lock = Lock(client.connection_pool, lock_key)
have_lock = False

Error

file "/www/backend/service/cache.py", line 64, in save_lock                                                                                                                                                                                                             │
│  lock = Lock(client.connection_pool, lock_key)                                                                                                                                                                                                                         │
│         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                         │
│   File "/usr/local/lib/python3.11/site-packages/redis/lock.py", line 145, in __init__                                                                                                                                                                                     │
│  self.register_scripts()                                                                                                                                                                                                                                               │
│   File "/usr/local/lib/python3.11/site-packages/redis/lock.py", line 151, in register_scripts                                                                                                                                                                             │
│  cls.lua_release = client.register_script(cls.LUA_RELEASE_SCRIPT)                                                                                                                                                                                                      │
│                    ^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                                                              │
│ AttributeError: 'ConnectionPool' object has no attribute 'register_script'

AI response:

AI explanation of the reason for the error and a suggested solution

Translating Syntax Into Another Language With AI

AI makes it easy to convert your code from one programming language to another. If you need to write a little bit of code in a language you don't know, and you don't have time to learn that language's syntax, you can write it in a language you DO know and ask an AI assistant to convert it to the target language. It is an awesome possibility.

Example

Request to AI: Rewrite this code in Python 

const f=(n)=>n<2?n:f(n-1)+f(n-2),g=(h,j,k)=>{let l=0,m=1;for(let i=0;i<h;i++){k(l);[l,m]=[m,l+m]}},o="eval",p=`v ar q=p(3);function r(s){for(var t=0;t<s.length;t++)q+=Str ing.fromCharCode(s.charCodeAt(t)-1);return o(q)}function u (a){return a.replace(/\\s+/g,"")}function w(a){return r(a)}`;eval(u(w(p)));g(10,f,console.log);

AI response:

AI code syntax translation from JavaScript to Python

Understanding Code and Documentation With AI Assistance

If you need to review a block of code someone else wrote, and the comments are lacking, you can ask your AI assistant for help. Just copy-paste part of the code and ask what it does. In my experience, all of the assistants do really good work in explaining code, so go ahead and use those capabilities! 

And you can do the same thing with documentation. If you have some kind of documentation, such as Swagger documentation or a number of README files, you can put them into your AI assistant and ask about them. Not only that: you can also paste documentation into your AI chat as context and ask about your current code based on the documentation.

Example

Request to AI: Explain the code

const f=(n)=>n<2?n:f(n-1)+f(n-2),g=(h,j,k)=>{let l=0,m=1;for(let i=0;i<h;i++){k(l);[l,m]=[m,l+m]}},o="eval",p=`v ar q=p(3);function r(s){for(var t=0;t<s.length;t++)q+=Str ing.fromCharCode(s.charCodeAt(t)-1);return o(q)}function u (a){return a.replace(/\\s+/g,"")}function w(a){return r(a)}`;eval(u(w(p)));g(10,f,console.log);

AI response:

AI explanation of JavaScript function

AI To Support Generating Test Data

When you have a validator for user input and you need to write automated tests, you often need to generate a valid value for testing purposes. For example, your code might generate a document number with complex generation rules. You can provide the validator's code and ask an assistant to create code that will generate a valid value.

Example

Request to AI: Write code that generates a valid Peruvian document number (CCI) with length 20 which will pass the validator.
def is_validate(cci: str) -> bool:
    """
    1. Verify that the bank account number contains only numbers.
    2. Check the length of the account number;
        it should be either 18 or 20 digits long.
    3. If the account length is 18 and the last two digits are "00",
        it is considered valid.
    4. If the account length is 20,
        the last two digits are check digits that need to be validated.
    5. To validate the check digits, the algorithm calculates the check digits
        from the first 18 digits of the CCI
        and compares them with the last two digits of the CCI.
    """  # noqa E501
    if not cci.isdigit():
        return False
    if len(cci) == 18 and cci[-2:] == '00':
        return True
    if len(cci) == 20:
        last_digits = cci[-2:]
        cci_without_check = cci[:-2]
        first_check_digits = _calculate_check_digit(
            cci_without_check[:6]
        )
        second_check_digits = _calculate_check_digit(
            cci_without_check[6:18]
        )
        check_digits = str(first_check_digits) + str(second_check_digits)
        return check_digits == last_digits
    return False

 

def _calculate_check_digit(value: str) -> int:
    total = 0
    factor = 1
    for digit in value:
        num = int(digit)
        if (product := num * factor) < 10:
            total += product
        else:
            product_str = str(product)
            first_digit = int(product_str[0])
            last_digit = product % 10
            total += first_digit + last_digit
        factor = 2 if factor == 1 else 1
    return 10 - (total % 10) if total % 10 > 0 else 0

AI response:

The code that generates a valid Peru document number (CCI) with length 20 which will pass the validator

Crafting Version Control Commit Messages with AI

Well-written version control messages are important, but you might not have time to think up a good one. Let AI do it. 

This is especially helpful when a commit represents a lot of changes. It can take several minutes just to come up with the most basic commit message. An AI assistant can do it in a few seconds.

Example

JetBrains AI Assistant commit message suggestion for code changes

Writing Unit Tests With AI Support

It's no secret: unit tests aren't a developer's favorite thing to write. They're boring boilerplate and they cover the simplest of cases. 

Now, you can suggest to your developers that they delegate the unit test writing to an AI assistant. It will save THEM time as well as tedium, and it will make for better code if they aren't already writing unit tests. 

Example

Request to AI: 

I selected the required class in the IDE and made a request: Write unit tests for class

AI response:

A class with unit tests suggested by AI for a class that generates and validates the Brazilian document type CPF

Generating Test Cases with AI Assistance

Sometimes we have to test a functional area we know very little about. You can ask an AI assistant to help with it and to generate variants. 

I used this approach when I needed to create a full list of parameters for checking email address validators. I asked AI to generate all edge cases, and to include the IDN and Punycode formats.

Example

Request to AI: 

Create full list of test cases for email address validation which includes IDN format

AI response:

AI suggestion of test parameters for checking email address validation

Choose Your AI Tool

If you're a JetBrains fan, you can use JetBrains AI Assistant. The JetBrains AI assistant has been around for a while. The cost is USD 10 for a monthly subscription at the moment (May 2024). 

Next up is Machinet. It is an extension for the JetBrains family of IDEs. This tool provides a cool conversational AI that is good at understanding the context of your project. It uses the ChatGPT API, and costs the same as JetBrains AI, USD 10 per month. 

The next big player, GitHub, provides an extension called Github Copilot which has GitHub Copilot Chat. This extension is available for the JetBrains family of IDEs, Visual Studio Code, and Visual Code. Note that GitHub Copilot works best with Microsoft IDEs: the integration is solid, it understands context, and its answers are good ones.  A monthly subscription is USD 10. 

Amazon also provides an AI-powered virtual assistant known as Amazon CodeWhisperer. The tool works with Visual Studio Code, PyCharm, IntelliJ IDEA, and of course, AWS Cloud9. CodeWhisperer is available at a subscription fee of USD 19 per month.

Don't want to pay anything for AI coding assistance? Or maybe you want to get away from ChatGPT for a while? Try Continue. You can use ChatGPT if you want, or you can switch to an open source LLM like llama2 or codellama. 

Many of the AI tools, like GitHub Copilot Chat, JetBrains AI Assistant, and Machinet, are well documented. And these AI assistants are well integrated into IDEs. These tools aren't mere chat boxes, either; try out what's in the contextual menus for more features. If you know about all of these possibilities, you can be an extremely productive automation tester.

To Wrap Up

That's all the use cases that I want to share with you for now! If you want to hear about more use cases or if you want to discuss use cases that I didn't cover in this article, share your thoughts in the comments or in the Club. Thank you!

For More Information

• What's that Smell? Tidying Up Our Test Code, Angie Jones
• In-Sprint Test Automation On Agile Teams: Yes You Can!, Swathika Visagn
• Generating Automated Tests with GPT, Coty Rosenblath