Generative AI-Based Testing Certification

Your Progress: out of 27

Validations

Validations are typically about something visible on the page. There are many different types of validations you can use with the “check” command.

check that page contains "error message"

You can also perform negative validations.

check that page doesn't contain "error message"

The keywords verify, assert, validate, and assess are all alternatives to check.

Many times, there are several instances of the element we want to validate and we need to pinpoint a specific one with relative locations. These work the same way with validations as with clicking.

check that page contains "error message" below "Password"

We can use multiple relative locations if necessary.

Sometimes, the string being validated changes dynamically each time the test case is executed. If the string is supposed to have the same pattern each time, we can use testRigor’s template system: For example, if we need to validate a phone number number, we can use the following:

check that page contains simple template "###-###-####"

If the pattern is not identical but similar with reasonable variations like the amount of digits in the string, we can use RegEx to validate. For example, if we need to validate a series number that has between 9 and 11 digits, we can use the following:

check that page has regex "[0-9]{9,11}"

testRigor can validate CSS properties of elements and divs such as text color, background color, font, etc

check that property of "background-color" of "my-div" is equal to "rgba(0,0,0,1)"

By default, testRigor uses soft validations, which means that a test case will not fail if the validation fails but will continue to execute steps until it is no longer possible or until the test case is completed. This was designed this way in order to allow users to collect as many issues with a single flow as possible and to prevent triggering a scenario where the case stops on a validation only to find that the next step is also a failure. However, it is possible to use hard validations by appending the phrase and stop test if fails to your regular validation.

Categories

check that "menu" contains "Request a Demo"

In addition to page-level, div-level, and element validations, testRigor allows validations to be performed directly against variables. However, since testRigor is always looking at the UI for validations, in order to perform a validation against a variable, we must use the keyword itself:

call api "https://testrigor.com" and save it as "variableName"
check that stored value "variableName" itself contains "James"

As shown in the example above, one of the most common instances where we see validations against variables is when validating API results. However, this can be used in any scenario where the validation or condition is based on what the variable itself is and not whether or not it is present in the UI. For example, the condition below is based on what the variable is instead of its presence or absence on the screen.

If stored value "variableName" itself is equal to "James" then
   enter stored value "variableName" into "First Name"
   click "Send"
end

check that url matches regex from the string with parameters "${homePrefix}/product/${type}/[0-9A-Za-z\-]+"

For full documentation and examples of variables, click here.

Screenshot behavior with validations

It is important to understand that individual validations do not prompt screenshots. A screenshot will be taken of a validation if it is the final step in a test case, but that is by virtue that it is the final step of the test case and not because of the validation per se. As a rule of thumb, screenshots are prompted on commands where the viewport is expected to change. See Click, Drag, Hover for prompting screenshots where they would not normally be taken.

AI Validations

In this section, we’ll explore various types of validations that can be performed using AI within testRigor.

Using AI to Validate Page Content

There are two main ways you can use AI for validation. Please note that AI models can occasionally produce incorrect results (hallucinations), so use these validations carefully.

General Page-Level Validation

Example:

check that page "contains a positive message in the chat response" using ai

In this case, "contains a positive message in the chat response" is treated as a prompt by the AI model. testRigor analyzes the page’s UI and source to determine whether the chat response indeed contains a positive message.

Element-Specific Validation

Example:

check that "element" "contains a positive message" using ai

Here, the AI focuses on validating the presence of a positive message within the specified element, rather than across the entire page. This is especially useful when similar text appears in multiple places, and you want to validate only one specific instance.

Using AI to Validate Custom Statements

You can ask AI to verify if a specific statement about the UI is true.

Example: 

check that statement is true "page contains testRigor logo"

Here, the entire string "page contains testRigor logo" is passed to AI, which checks whether the statement is true by comparing it against the current UI and page source.

If the statement is not true, testRigor provides detailed reasoning for the validation failure.

Example Error:
The statement ‘page contains testRigor’s classic logo’ is false because, based on the screenshot and page source provided, there is no classic testRigor logo visible on the page. The header shows “tR Playground” text as the logo in the top left, and there’s a “Welcome to testRigor Playground” header in the top center, but no classic testRigor logo appears anywhere on the visible interface. The page shows a collection of QA automation challenge cards but doesn’t contain the classic testRigor company logo.

Screenshot provided for reference:

Using AI to Detect UI Errors

testRigor can also analyze the page visually to detect UI issues.

Example:

check page for UI errors

You don’t need to explicitly write using ai in this case. testRigor automatically understands that AI-based analysis is required here. It analyzes the screenshot of the page to detect any visual/UI issues.

Customizing Error Severity

You can control what severity level of issues are reported or how they’re treated.
Example 1:

check page for UI errors reporting major errors or higher

Only errors of major or higher severity will be reported.
Example 2:

check page for UI errors treating errors as major or lower

All reported errors will be treated as major or lower..

Example 3:

check page for UI errors reporting major errors or higher treating errors as major or lower

Errors of major or higher severity are reported and treated as major or lower.

Example 4:

check page for UI errors treating errors as major or lower reporting major errors or higher

testRigor treats the errors as major or lower, but still only reports those which are major or higher in severity.

Example Output:

Errors: 

MAJOR UI Error: Sign-in prompt overlaps with the website content

MAJOR UI Error: Location permission request popup overlaps with the website content

See the section on scrolling for information about how auto-scrolling affects validations.