Certification: AI-driven Test Automation

AI and Prompt Engineering

AI and Prompt Engineering

Generative AI facilitates direct AI interaction with a UI based on straightforward prompts. We instruct the AI on the workflow (as described) or segments of it (using AI-based rules), and the AI then extracts data from the page, striving to execute the task autonomously.

You can use this feature to either:

Generate a sample test with AI
Generate an actual test with AI

In both cases, the process of test creation is as straightforward as providing a title or description of the test case, and clicking on the respective buttons. If you opt for the sample test, you will receive a series of steps in the correct format, which you can then tailor to your specific needs to create a test case. On the other hand, if you opt to generate an actual test with AI, testRigor will generate a comprehensive test based on your application.

After generating the test, you have the option to modify values, add more assertions, or make any other necessary adjustments. This flexibility allows you to tailor the test case to perfectly suit your specific requirements.

Watch the video for a detailed step-by-step guide on how to use this feature.

Test Case Generation (Corner Cases)

There is an ability to automatically generate corner cases for any given test case.

  • The cases generated will skip actions that enter/type data into fields one by one and find the errors associated with the step that has been skipped.
  • The Generate corner cases button is located on the information bar of each test case on the left of the Link data set link.
  • Cases that have been generated will not have the option to generate other corner cases. They will have a label or text indicating that they have been generated as corner cases instead.

AI-based Rules

You can write free-flowing sentences in the “Custom steps” multiline edit box like: “Find and select a Kindle.”

If the rule doesn’t produce the sequence of steps you anticipated, there are always two methods to address it:

  • Prompt Engineering
  • Fixing the steps

Prompt Engineering

You can modify the name of the rule to better convey what the AI should accomplish and provide more clarity there. From an AI standpoint, the rule’s name acts as a “prompt,” directing its actions. Occasionally, providing more context can help.

For example:
find a Kindle
may not perform as you’d expect. The AI might simply enter the word “Kindle” in the search bar and cease execution once it observes the dropdown with Kindle options, interpreting it as having already found the Kindle. A potential solution might involve directing it to click on a Kindle to navigate to the product page, like so:
find and select a Kindle

Fixing the Steps

Alternatively, you can directly adjust the steps. Navigate to the Rules section on the left, locate the rule, and deselect the “AI” checkbox, which will then reveal the underlying steps.

In this scenario, you can add a step like “click ‘Kindle Paperwhite'” to ensure progression to the subsequent phase. Generally speaking, you can modify the steps as needed. This action transforms the rule into a regular one, bypassing the AI and strictly following your designated sequence of steps. Still, some instructions you incorporate could be AI rules in their own right.

When to use Prompt Engineering vs. Fixing the Steps

Utilizing AI is advantageous because it can elevate your tests to new levels of stability by amplifying the abstraction level you use for descriptions. For instance, “find and select a Kindle” is indifferent to the intricacies of your search system and the exact steps needed for execution. The trade-off is that AI-based operations may be somewhat slower, and the AI might occasionally misinterpret your intentions unless you provide exhaustive details.

If your objective is to verify that a specific feature operates in a manner detailed in the specification, you might opt for explicit commands. This ensures the test flow aligns perfectly with the specified requirements and will fail if any step falters.

We believe that the best application of AI is its conjunction with standard testRigor steps in scenarios where intricate details are excessive. For example, on a list of Kindles, if you aim to select any model, you wouldn’t want to rely on specific CSS/IDs, especially if the term “Kindle” appears frequently on the page. In such a situation, you might use a directive like:
choose a specific Kindle

This would allow the AI to autonomously select any of the actual products.

As a guiding principle, employ AI when the details of achieving or performing a step are inconsequential and unlikely to change. Conversely, use specific steps when verifying that a functionality operates in a specific manner or adheres to specific terminology.

Watch the video tutorial here:

Test your knowledge

There is no way to do it with AI.
I want a TV, Samsung 80”, please!
find and select 80” Samsung TV
checkout 80” Samsung TV

There is no difference; the format is the same, and both can be edited and expanded by hand.
AI-generated test cases cannot be edited.
AI-generated test cases have more assertions than the manually created ones.
AI-generated test cases are shorter and less comprehensive.

To automate QA testers out of their jobs.
To speed up initial test creation, making it more straightforward to expand and edit according to the desired result.
They only focus on the most critical parts of an application.
They are always 100% accurate in identifying bugs.