My Certificate Find a Certificate About Certification QA Terminology Basics of Software Testing Basics of Test Automation QA Processes Overview Basics of testRigor BDD & SDD Web Testing Overview Mobile Testing Overview Native Desktop App Testing Overview API Testing Overview Click / Drag / Hover Entering Data Variables Validations Scrolls and Swipes Dealing with Dates and Times Dealing with Emails Working with Tables Conditional Sequences Cycles & Loops Reusable Rules SMS and Phone Calls Managing Multiple Environments Settings Integrations AI and Prompt Engineering Final Exam

Generative AI-Based Testing Certification

AI and Prompt Engineering

AI and Prompt Engineering

Generative AI’s main benefit is its ability to speed up initial test case creation. It facilitates direct AI interaction with a UI based on descriptive prompts. The first and most important prompt is the description of the application to be tested. Descriptions can be provided when the test suite is being created, or they can be added later in Settings->Test generation->AI-based Test Generation. The more detailed the description is, the better chance that generative AI will be able to build successful tests.

Poor description for a banking app:

This is a banking app. It does the stuff a banking app usually does, but this bank is the best. Please be smart and make test cases based on the information that only I know about my application.

Good description for a banking app:

This is a banking application for users on desktop web browsers. Users must be logged in to perform transactions. Users can view balances, inquire about transaction history, perform fund transfers, make bill payments, access financial statements, access credit scores. Users can also apply for loans, view loan status, and use credit or mortgage calculators to estimate mortgage payments, affordability, and refinancing options. In addition, users may also use the platform to browse investment and trading opportunities. It has portfolios and research tools to access market news, analysis, and financial research reports as well as the ability to set up alerts for specific investment opportunities such as price movements, dividend payouts, and other market events. Finally, users, can leverage tools for financial planning to help them set up budgets and goals.

The next step in using generative AI successfully is providing workflows. testRigor can provide generic suggestions for workflows, but user-generated prompts are best. These prompts can be through the test cases description and through smaller segments called AI-based rules. The AI then extracts data from the page using the prompts in an attempt to execute the task autonomously.

Description-only prompts will allow you to do either of the following:

Generate a sample test with AI 
or
Generate an actual test with AI

In both cases, the process of test creation is as straightforward as providing a title or description of the test case, and clicking on the respective buttons. If you opt for the sample test, you will receive a series of steps in the correct format, which you can then tailor to your specific needs to create a test case. On the other hand, if you opt to generate an actual test with AI, testRigor will generate a comprehensive test based on your application.

After generating the test, you have the option to modify values, add more assertions, or make any other necessary adjustments. This flexibility allows you to tailor the test case to perfectly suit your specific requirements.

Watch the video for a detailed step-by-step guide on how to use this feature.

AI-based Rules

You can write free-flowing sentences in the “Custom steps” multiline edit box like: “Find and select a Kindle.” testRigor will

If the rule doesn’t produce the sequence of steps you anticipated, there are always two methods to address it:

  • Prompt Engineering
  • Fixing the steps

Prompt Engineering

You can modify the name of the rule to better convey what the AI should accomplish and provide more clarity there. From an AI standpoint, the rule’s name acts as a “prompt,” directing its actions. Occasionally, providing more context can help.

For example:
find a Kindle
may not perform as you’d expect. The AI might simply enter the word “Kindle” in the search bar and cease execution once it observes the dropdown with Kindle options, interpreting it as having already found the Kindle. A potential solution might involve directing it to click on a Kindle to navigate to the product page, like so:
find and select a Kindle

Fixing the Steps

Alternatively, you can directly adjust the steps. Navigate to the Rules section on the left, locate the rule, and deselect the “AI” checkbox, which will then reveal the underlying steps.

In this scenario, you can add a step like “click ‘Kindle Paperwhite'” to ensure progression to the subsequent phase. Generally speaking, you can modify the steps as needed. This action transforms the rule into a regular one, bypassing the AI and strictly following your designated sequence of steps. Still, some instructions you incorporate could be AI rules in their own right.

When to use Prompt Engineering vs. Fixing the Steps

Utilizing AI is advantageous because it can elevate your tests to new levels of stability by amplifying the abstraction level you use for descriptions. For instance, “find and select a Kindle” is indifferent to the intricacies of your search system and the exact steps needed for execution. The trade-off is that AI-based operations may be somewhat slower, and the AI might occasionally misinterpret your intentions unless you provide exhaustive details.

If your objective is to verify that a specific feature operates in a manner detailed in the specification, you might opt for explicit commands. This ensures the test flow aligns perfectly with the specified requirements and will fail if any step falters.

We believe that the best application of AI is its conjunction with standard testRigor steps in scenarios where intricate details are excessive. For example, on a list of Kindles, if you aim to select any model, you wouldn’t want to rely on specific CSS/IDs, especially if the term “Kindle” appears frequently on the page. In such a situation, you might use a directive like:
choose a specific Kindle

This would allow the AI to autonomously select any of the actual products.

As a guiding principle, employ AI when the details of achieving or performing a step are inconsequential and unlikely to change. Conversely, use specific steps when verifying that a functionality operates in a specific manner or adheres to specific terminology.

Watch the video tutorial here:

Test Case Generation (Corner Cases)

There is an ability to automatically generate corner cases for any given test case.

  • The cases generated will skip actions that enter/type data into fields one by one and find the errors associated with the step that has been skipped.
  • The Generate corner cases button is located on the information bar of each test case on the left of the Link data set link.
  • Cases that have been generated will not have the option to generate other corner cases. They will have a label or text indicating that they have been generated as corner cases instead.

Test your knowledge

There is no way to do it with AI.
I want a TV, Samsung 80″, please!
find and select an 80″ Samsung TV
checkout 80″ Samsung TV
Regular test cases depend on the accuracy of steps in the test case editor and rules while generative AI relies on the clear description of the prompts it is given to work with.
AI-generated test cases cannot be edited.
AI-generated test cases have more assertions than the manually created ones.
AI-generated test cases are shorter and less comprehensive.
To automate QA testers out of their jobs.
To speed up initial test creation.
They only focus on the most critical parts of an application.
They are always 100% accurate in identifying bugs.
© 2023 testRigor. All rights reserved.