Getting Started guide
Introduction to testRigor
testRigor is an AI Agent that allows anyone to create end-to-end tests from an end user's perspective using plain English, therefore eliminating excessive test maintenance related to locator changes. testRigor supports testing on the following platforms:- Web testing (Windows, MacOS, Ubuntu) and Mobile Web testing on iOS and Android
- Native and Hybrid Mobile App testing for iOS and Android
- Native Desktop applications testing
- Acceptance testing
- Smoke testing
- Regression testing
- System (end-to-end) testing
- API testing
- Visual testing
- SMS and phone call testing
- 2FA and Captcha testing
- Leverage testRigor's Generative AI to create tests based on descriptions
- Write tests from scratch using plain English commands (See this documentation for help)
- Use testRigor's record-and-playback tool
Setting Up an Account
testRigor can be highly customized according to your needs. For the most recent pricing information, please contact our friendly sales team. Here are some options to consider:- Cloud (default) or on-premise
- Windows, MacOS, Ubuntu, iOS, Android, the number of devices, etc.
- Number of parallelizations for faster test execution speed
Creating Your First Codeless Test Cases
Tutorial for creating your first test case:
All video tutorials: https://testrigor.com/tutorials/
All recent public test cases https://app.testrigor.com/public-tests.
See more details on step-by-step test creation here.
testRigor English-language support documentation
The goal of testRigor is to allow you to write your tests in your way of saying it in plain English. It is extendable by allowing you to support your own phrases. And you might want to get familiar with test automation best practices and, more importantly testRigor Best Practices.
You can click on elements with
click "Submit"
, enter data with enter "Peter" into "First Name"
and validate with check that page contains "Welcome, Peter!"
.Table of Basic Commands
We support a vast variety of commands. The list below contains many examples of how to express them.
Action | Options | Example |
---|---|---|
click |
double, triple, right, middle, long, times, in a context of, using the mouse, using javascript, without scrolling, using OCR, using OCR only, and switch to the new tab |
or:
or:
or:
or:
or:
or:
or:
Note: You can use middle click to open links in a new tab
or:
or:
For web testing you can specify the way we click on an element as follows:
or:
Note: In headless mode this option will be ignored as the only method available now is javascript
or:
Note: You can use "without scrolling" to prevent testRigor's default behavior (scroll to the element before the click).
or:
Note: This sample uses an image stored with the name "Logo" in "Test Data". The image is used to identify the location of the click. For example, this type of click is useful when you need to click on an image map. The default discrepancy is 20%.
We also support mobile-specific commands to go to the home screen or review recently used apps.
To click at Home Button is available to mobile Android and IOS using the commands:
To click the Recent Button is available on mobile Android using the following commands (Android only as iOS doesn't have this button):
There is also a command to switch to the new tab when the click action automatically opens one.
|
generate |
Simple Template, unique email, unique name, RegEx, Google authenticator code |
Generate a unique email in testrigor-mail.com domain:
Generate a unique name:
Generate a unique phone number:
Generate a unique string of letters/numbers:
Generate unique the date and time with unique parameters:
Generate an email address in a custom domain:
Generate multi-line:
Generate and type:
Generate and save only:
Generate Google authenticator time-based one-time password (TOTP) code for 2-step or multi-factor authentication. It needs the user QRCode saved in a stored value or the user text secret to generate the code:
|
enter |
enter, tab, escape, delete, backspace, ctrl+a, arrow right, arrow left, arrow up, arrow down, go, search, send, next, previous, done |
Note: Enter supports selects/dropdowns/checkboxes/etc.
For checkboxes, you can do the following:
For enter, tab, escape, delete, backspace, ctrl+a, arrows or Android-specific go, search, send, next, previous, done:
For multi-line inputs:
For selects or dropdowns:
*Note: Avoid using the "enter into" command when the cursor is already focused on the desired field. For these cases, use the "type" command instead.
It is also possible to select an option by position in selects or dropdowns:
*Note: Due to the differences required for testRigor to interact with desktop applications (i.e., browsers) versus the web application UI, the "select" commands were created in order to interact with native browser dropdowns/selects that are tagged as <select> in the dom. If your dropdown is a custom dropdown, it is preferable to interact with it as a manual user would (i.e., via clicks, scrolls, etc). |
type |
enter, tab, escape, delete, backspace, ctrl+a, arrow right, arrow left, arrow up, arrow down, go, search, send, next, previous, done |
It should be mainly used for typing free text into input fields. You may type text without referencing an input field if the previous step has already placed the cursor within the input field you wish to use:
For multi-line inputs:
Press keyboard keys:
Or you can press arrow keys like this:
|
copy and paste |
text |
Copy selected text (highlighted by either double clicking or dragging the mouse) to the clipboard:
Copy text from string (non-headless only):
Copy text from saved value (non-headless only):
In order to paste copied text click on the input field and use one of 2 commands:
or simply
|
check |
page |
Check page content by string:
or
Check page content (including the OCR recognized texts) by string:
Check page content using OCR recognized texts only:
Check that page did not change after some action:
Note: This action compares page images pixel to pixel
Check that page doesn't have a fourth button:
Validate page with Vision AI:
Look for UI/UX issues using Vision AI:
Priority options are: minor, major, critical, blocker |
check |
element |
and
You can also validate whether an input/checkbox/button/etc. is disabled, clickable, or enabled
Check stored value itself:
Check that an element looks the same as it did on the same step during last successful run:
Check that the entire screen looks the same as during last successful run:
Check that SMS is received:
Check that file is downloaded and check its content:
You can also reuse the downloaded file or multiple files to upload them later in the test. For this you need the following action:
The variable "DownloadedFiles" will contain all files downloaded after the last check for downloaded files. Then to upload the file on some other page, just use the following action
You can also extract text from a downloaded file and save it as a variable.
Check that a container contains a nth element:
Check that an element is changing. This is useful to check that some element that contains animation or video is updating (desktop web browser testing and Android only):
You can also use a negative check:
Validate element with Vision AI:
|
check |
statement is true/following statement is correct |
Utilize Vision AI tovalidate statement about the page
|
long press |
|
|
hover |
|
|
open new tab |
|
|
switch tab |
or
Note: In order to easily keep track of tab numbers, testRigor's default setting is to begin each test case in a new browser window. Therefore, tab 1 is always the tab the test case starts in, and tab 2, 3 and so on are the new tabs in the order that they are opened throughout the test case run. Note that quotation marks are not needed for tab numbers. Quotation marks are needed for tab names only. For pop-ups, instead of a tab number, you will need to use the popup window name or title. |
|
close tab |
|
|
go back |
|
|
go forward |
|
|
reload |
|
|
reset to home |
|
|
restart app |
Restarts application without clearing data.
Note: This action is for mobile devices only. |
|
drag |
element, file |
Or file:
Or draw on a canvas:
Drag mouse to multiple points without releasing the click:
If you need to drag a folder with files, you need to zip it and upload in Test Data section and use it like so:
Note: This action does NOT work in headless mode. It also does NOT work with Internet Explorer |
scroll |
down, up, left, right |
Or:
Partial scroll:
You can also scroll directly to a specific part of the page:
Or:
Using the mouse:
For pages with several scroll areas, you can also scroll up or down on a specific portion of text using the mouse wheel:
If the target text is very far from the starting point of the scroll, you can focus scrolls on the area of the text instead of the text itself by using mouse wheel action in a loop:
Important Note: The scroll until page contains method works best in Visible first mode. Visible first prioritizes only what is visible in the viewport/screenshot. Batched mode opens testRigor's visibility to anything that is loaded on the page whether it is visible to the end user or not. (To toggle this setting, you can find the dropdown in Settings->Speed optimizations->Performance->Getting visibility of elements approach) |
swipe |
down, up, left, right |
Or:
|
wait |
time |
Note: 2 min max |
call api |
URL |
Or with more parameters:
|
mock api call |
URL, returning (headers, body or http status code) |
Or with more parameters:
|
grab value |
Simple Template, RegEx, element, by description(using ai) |
or, to grab a value from
UserName:[email protected]
or:
or, to grab a value using simple template:
or:
or, to grab a value from the page using AI:
or, to grab a value from an element using AI:
You can also grab multiple values from a table row or column (the value will be stored as a JSON Array)
|
extract value by regex or template |
element |
For example, to extract a value of username from variable
var1 containing UserName:[email protected] you could use:
For example, to extract the ID number from variable
var1 containing userID:123456 , you could use:
|
save value |
|
|
store clipboard |
Clipboard current value can be saved into a variable for later usage:
|
|
open url |
URL |
Note: The domain must be whitelisted! |
grab url |
You can grab the browser's current URL and save it into a variable:
|
|
download file |
URL, username, password |
You can download a file by URL for further actions (as "check that file", "enter" or "upload file" commands):
HTTP, HTTPS, FTP, FTPS and SFTP protocols are supported.
Authentication information can be embedded into URL
or specified separately
|
upload file |
device, template, URL, username, password |
You can upload a file to URL (FTP, FTPS and SFTP protocols are supported):
As a source can be used file from test data, previously saved file or remote file by URL. You can also generate a new file based on a template:
You can upload a file into currently active testRigor mobile device:
In this case, the file will be uploaded into Downloads folder on the Android device. |
send email |
Here is how to send a simple email:
To send an email with an attachment, you can upload the file into testRigor store in "Test Data" section, then use it by name as stored value. Alternatively you can use your own URL. If you choose to attach a file from your own URL, the link should be downloable.
|
|
check email |
Or
By default testRigor renders emails in a desktop browser. If you want to render it in the mobile device (available for Android and iOS), you need to add the following suffix:
and show in mobile E.g.:
You can use both - sender and recipient
By default we assume that the user expects one email message. Multiple emails will trigger an error message. It is possible to customize this behavior by specifying exactly how many emails are expected or just say "one or more".
Also it is possible to filter messages by subject
Notes:
|
|
reply to email |
Or
Note: this command will render the received email (not the response) as if it is a web page |
|
call |
phone number |
Or, simply:
|
sms |
phone number |
Or
|
set geo location (GPS Coordinates) |
|
|
start browser / start device |
name |
Starts a new browser with its own new session.
Start and switch:
For device:
|
start browser with cert |
name, filename or user |
Starts a new browser with client certificate pre-selected by file name.
Start and switch:
Pre-select certificate by subject CN:
Note: Certificate pre-selection is only available for Google Chrome Browser on Windows |
switch to browser / switch to device / switch to remote desktop |
name |
For device:
Get back to the first browser:
For remote desktop:
Note: This command is only available for Windows Remote Desktop suites. It will return to the first remote desktop if the user has switched to browser at some point during the test case execution. |
switch context to native / switch context to browser |
Switch the context of the following actions to the entire mobile device (as for application testing) or start the mobile browser and use only its content (as for web testing). |
|
compare |
type of element |
Compares current elements on the current screen with the ones appearing the previous time the
screen was shown.
The following sentence compares element on the current test run screen with the one appearing on the reference screen or saved as a file pixelwise. If the element appears to be different, both baseline and current (failed) run element images will be saved in the artifacts for manual investigation.
The sentence must contain an image definition, such as
image of text "pressed" , image of 3rd "peter" , etc. Optionally it may contain an allowance value in percent. If missing, then it is assumed to be zero. If element size changed, centered intersection of the element is compared. Pixels are compared using mean squared error.Silimarly you can make a screenshot of an image, save it, and then use as a reference like so:
|
login |
Complex action. Identifies and performs the necessary steps to login. |
|
crawl sitemap / go through sitemap |
testRigor supports XML and Text sitemap formats.
|
|
execute JavaScript in the browser |
testRigor supports execution of vanilla Javascript in the browser.
|
|
accept alert/prompt |
prompt value |
Browser alert/prompt can be accepted with a value:
|
change device orientation |
portrait, landscape |
Device orientation can be changed to portrait/landscape on android/iOS device:
|
zoom |
in, out, open, close |
In native mobile testing (android/iOS), zoom can be applied using the pinch gesture:
The zoom value is set to 50% by default, and it is applied from the center of the screen. It is possible to specify the screen or element from which the motion will depart, as well as an offset.
Another syntax option is to use pinch open/close instead of zoom in/out; both methods function the same.
|
open deeplink |
URL, package or bundle |
In mobile testing (web and native), deeplinks can be opened passing the URL:
For package/bundle information, you can pass that explicity to the command.
Example: opening a deeplink to a specific place using the "Maps" app on Android:
For Android, if no package information is provided, then the default application package will be used.
Example: opening up accessibility options inside the settings app on iOS 17:
For iOS, if no bundle information is provided, then the default application for the given url scheme is going to be used. |
install application |
alias of mobile application |
Extra mobile application can be uploaded under Settings->Multiple applications and installed on testRigor android/iOS device.
|
hold key, release key |
Control, Shift, Alt, Command, F1, F2,...F12, Enter, Space, Delete, Backspace |
Hold key could be use in combination with a click.
Example of use of Hold and Release
|
set camera image to |
URL |
Set custom image for camera. Works only for Sauce Labs provider on Android Real Devices.
JPG, JPEG and PNG files with size up to 5 MB are allowed.
You can also provide image file as stored value:
|
enable touch ID/face ID, disable touch ID/face ID |
In native mobile testing (iOS ONLY), biometrics can be enabled by the command
for devices that provide Touch ID technology, and
for devices that provide Face ID technology.
The same applies for disabling biometrics
This command is related to scan using. |
|
scan using |
touch ID and pass, face ID and pass, touch ID and fail, face ID and fail |
Respond with a valid/invalid match when biometrics is requested by native iOS app.
For devices supporting Touch ID, you can use command to simulate a valid biometric, by using:
and for devices supporting Face ID:
The same could be achieved if you need to simulate an invalid biometric. For Touch ID:
and for Face ID:
Before using this command, you need to be sure to enable biometrics for your device. (see enable touch ID/face ID, disable touch ID/face ID command) |
Reusable Rules (Subroutines)
If you have a sequence that will be used often, you can save them as a Reusable Rule and refer to it with the name of your choosing.
Example: The first 6 steps of the example below will take you to your checkout page. If you frequently perform this process, create one rule to simulate all of those steps.
login
click "Men Clothing"
scroll down
click "Men's cargo shorts"
click "brown"
click "Size 34"
click "Add to cart"
check that page contains "Your order is nearly complete!"
Then, go to the Reusable Rules section in testRigor, assign the name "go to checkout page" to your rule, and add the first 6 steps above. From then on, you can simply use the syntax below to trigger all 6 steps:
go to checkout page
check page contains "Add to cart"
You can also create rules with dynamic parameters.
Rule name:
search "product" click on the link and then press "button"
Steps: (We're going to create variables with the same name you define between quotes.)
enter stored value "product" into "search"
click link stored value "product"
click stored value "button"
Then you can call the rule like:
search "Computer" click on the link and then press "Add to Cart"
The variables defined are scoped and can only be accessed inside the rule.
Referencing Locations and Elements
All references must be in double quotes, which can be escaped by backslash. There are several ways to refer to elements (for checks, clicks, entering data, etc.).:
Attributes
The following list contains the supported attributes that testRigor can interact with. Attributes are searched for without the need to refer to them in the script. The command
check that page contains "peter"
 will search through elements with the following attributes:Attributes for Desktop web browser and mobile web browser testing
Desktop web browser and mobile web browser testing support the following attributes:
- Nested text - example
- Placeholder - example
- Value - example
- data-tid/data-testid/data-test-id/Id/resource-id/data-id - example
- Name - example
- aria-label - example
- CSS Class - example
- Label from ML classification - example
- Hint/Title/Tooltip - example
- Alt/Src - example
- For inputs/edits/dropdowns/selects/etc. will also search corresponding label - example
*Note: In addition to the above mentioned attributes, custom attributes can be added for desktop web browser testing in Settings => Advanced => Custom attributes to consider for finding elements new line separated.
Attributes for native mobile application testing
Native mobile applications support the following attributes:
Android
- content-desc
- class (usually named something like
android.widget.TextView
) - resource-id
- text/label
iOS
- accessibility-id
- XCUIElementType
- name
- text/label
*Note: Hybrid applications tested on testRigor infrastructure use the attributes both from web browser applications and native mobile applications.
Attributes for remote desktop application testing (only available if Windows Application Driver is properly set up)
- AutomationId
- Name
Generic Indexes
Generic indexes are allowed for when multiple instances of the same element are on the page. For example:
check that second "peter" color is "ffaabb"
check that 2nd "peter" color is "ffaabb"
example
Types
Types allow us to differentiate elements that have the same name on the screen. For example, if there is both a button and an input placeholder named "Search" on the page and you want to click the button, the way to specify it would be
click button "Search"
. Supported types are:- text - example
- label - example
- button - example
- link - example
- input (or "edit" or "field") - example
- dropdown (or "select") - example
- checkbox (or "switch") - example
- radiobutton - example
- file input (or "input file" or "edit file" or "input type file"): specifically an input of type file - example
Typed indexes
Typed indexes are a combination of general indexes and types. For example,
check that second input "peter" color is "ffaabb"
will find the second input/edit named "peter".Stored value for controls
Stored value for controls is the use of a variable to find and element or control. For example,
check that stored value "peter" color is "ffaabb"
will resolve the stored value for "peter" and use that resolved value to find the control.Image class
Image class uses image recognition technology to identify common icons. For example,
click on "cart"
. testRigor classifies images on the screen and if there is a button which looks like a shopping cart it will click on it. Complete list of all button image categories is here.Stored value for data
For example,
check that "peter" contains stored value "actionNote"
will find the second input/edit related to "peter".All the ways to refer an element described above apply to all types of actions like check, enter, click, etc. The options above can be combined together.
Multiple references support
Multiple references allows us to use
or
in instances where the targeted element has two possible names. In certain cases, certain elements might be expected to have different names but mean the same thing. For example:click "checkout" or "submit"
It can also be combined with "if exists" to avoid failing if the element doesn't exist:
click "checkout" or "submit" if exists
Selecting elements in reference to other elements
You can select elements based on it's position in relation to some other element (otherwise known as the "anchor"). For example, you can refer to elements in certain sections like:
click on button "Delete" below "Actions"
Supported relative locations are:
to the left of
- exampleto the right of
- exampleabove
- examplebelow
- exampleon the right top of
- exampleon the left top of
- exampleon the right bottom of
- exampleon the left bottom of
- examplenear
- example
You can also refer to the elements by 2 references:
click on button "Delete" below "Section Name" to the right of "label"
By default, testRigor will consider elements that are located at least 30% between the the target area (the yellow lines that extend from the anchor on screenshots). When the element is located less than 30% between the target area, certain keywords can be used to specify how the relative position should be used:
roughly
- used to search for an element anywhere on the screen in the direction of the relative location specified starting from the anchor (e.g., anywhere below the anchor for "below", anywhere to the right of the anchor for "to the right of") - examplewith at least "10" percent overlap
- used to specify when less than 30% of the element falls between the target area - examplecompletely
- As the first relative location in a command defaults to "roughly" when there multiple anchors, this keyword allows the first relative location to retain it's default meaning (i.e., it is only necessary when more than one anchor is specified and you need to keep the default meaning of the relative location) - example
For example:
enter "Peter" into roughly below "Section"
enter "Peter" into element with at least "1" percent overlap on the right of "Description"
However, if there are 2 anchors specified, the first anchor defaults to "roughly" and the second one uses the default behavior. For example,
enter "Peter" into "Section" below "Type" and on the right of "Description"
is equivalent to:
enter "Peter" into "Section" roughly below "Type" and on the right of "Description"
however, if you want "below" to keep its default meaning, you would need to say:
enter "Peter" into "Section" completely below "Type" and on the right of "Description"
Selecting elements in the context of other elements
You can select elements within the context of other elements. For example, you can pinpoint row at the table and ask to click a button in the context of that row:
click on "Delete" within the context of table "actions" at row containing "id1" and column "Actions"
You can use references to other elements (previous section) in the context itself
click on "Delete" within the context of "sectionTwo" below "Actions" and to the right of "rowName"
Selecting elements using reference to other elements and in the context of other elements
You can narrow where to look for an element using a combination of references to and in the context of other elements.
- Context is processed first, narrowing what to consider to what is inside that element
- References to other elements is processed next, only considering what was delimited by the context
click on "Button" below "Title" in the context of "sectionOne"
click on "Button" below "Title" and to the right of "leftHeader" in the context of "sectionTwo"
You can even specify up to 4 references to other elements using this combination
click on "Button" below "Subtitle" and to the right of "leftHeader" in the context of "sectionOne" below "Title" and above "Second Title"
Specifying the type of an element
You can force the system to only deal with a certain type of elements. The following command will not click on any text that is not a button or link:
click on strictly button "Delete"
Case sensitivity and exact matches
You can choose to deal with only "exactly" matches (i.e., case sensitive full strings match, or just say "case sensitive" to make matching case sensitive):
click on exactly "Delete"
It can be combined with the strict type selection like so:
click on strictly button exactly "Delete"
All supported modifiers to a string being searched:
case sensitive
exactly
(case sensitive full string must be exactly the same)full string
(not necessarily case sensitive)contains template
contains substring which matches a simple templatecontains regex
contains substring which matches a regular expressionmatches template
the full string must match a simple templatematches regex
the full string must match a regular expressionstrictly less
lexicographically strictly earlier (empty strings are always not less)less or equal
lexicographically earlier or equal (empty strings are always not less or equal)strictly more
lexicographically strictly later (empty strings are always not more)more or equal
lexicographically later or equal (empty strings are always not more or equal)
Fine-tuning access
In some cases, the target might be too wide, in this case you can force it to go deeper:
click deepest element "Delete"
It will issue click on the deepest element in the element tree that has the text.
You can also to force it go shallow elements. It is useful when the "Prioritize the deepest element" setting is enabled and we don't want to click on the deepest element:
click enclosing element "Delete"
It will issue click on the enclosing element in the element tree that has the text.
Specifying position on screen
It is possible but highly discouraged (for stability reasons) to use offsets to click on specific part of an element:
click on "Delete" with offset "20,10"
offset is calculated from top left corner of the element, horizontal coordinate first. It is also possible to specify some generic positions like:
click in the middle of the screen
The supported positions are:
in the middle of the screen
- examplein the top quarter of the screen
- examplein the second top quarter of the screen
- examplein the bottom quarter of the screen
- examplein the second bottom quarter of the screen
- examplein the top third of the screen
- examplein the bottom third of the screen
- examplein the left quarter of the screen
- examplein the second left quarter of the screen
- examplein the right quarter of the screen
- examplein the second right quarter of the screen
- examplein the left third of the screen
- examplein the right third of the screen
- examplein the left side of the screen
- examplein the right side of the screen
- examplein the top of the screen
- examplein the bottom of the screen
- exampleon the left edge of the screen
- exampleon the right edge of the screen
- exampleon the top edge of the screen
- exampleon the bottom edge of the screen
- examplein the left top corner of the screen
- examplein the right top corner of the screen
- examplein the left bottom corner of the screen
- examplein the right bottom corner of the screen
- example
Saved values (variables) support
To use a variable instead of an explicit string, add "stored value" or "saved value" before the variable:
validate that stored value "actionNotes" color is "ffaabb"
check that "peter" contains stored value "actionNote"
click on stored value "actionNotes"
check that page contains stored value from "actionNotes"
example
testRigor supports generating random values based on RegEx , saving them and using them later in the test. For example:
generate from regex "[a-z]{10,18}", then enter into "Notes" and save as "actionNotes"
There are 2 special stored values:
"username"
and "password"
which come from Application-specific credentials settings for your test suite. With it you can do:enter stored value "username" into "username_field"
enter stored value "password" into "password_field"
These stored credentials are also used by
login
command. Also, you can use stored variables as parameters in most commands by adding a $
and curly brackets (${variableName}
):enter from the string with parameters "${homePrefix}/my/path" into "urlPath"
*Note that using
$
and curly brackets with keywords string with parameters
allows users to concatenate or join variables with variables or variables with static values in the portion between quotation marks.Calculations and equations
And calculate expressions like this:
check that expression "${a} + ${b}" itself is equal as a number to "42"
or like this:
save expression "${a} + ${b}" as "answer"
testRigor supports ECMAScript 5.1 compatible expressions.
For example, you can calculate to check that a year from 30 days ago is present on the screen:
check that page contains expression "var aDate=new Date();aDate.setDate(aDate.getDate()-30);''+aDate.getFullYear()"
Pre-defined Variables
testRigor supports the following pre-defined saved values:
- username -
enter stored value "username" into "Email"
example - password -
enter stored value "password" into "Password"
example - homeDomain - (from URL in Test Suite Details: www.example.com)
open URL from string with parameters "https://${homeDomain}/cart/checkout/confirm"
 example - homeFile - (from URL in Test Suite Details: /cart/checkout/confirm)
open URL from string with parameters "https://www.example.com${homeFile}"
 example - homePrefix - (from URL in Test Suite Details: https://www.example.com)
open URL from string with parameters "${homePrefix}/cart/checkout/confirm"
 example - todayYear - (2023)
enter stored value "todayYear" into "YYYY"
example - todayYearShort - (23)
enter stored value "todayYearShort" into "Year"
example - todayMonthNumber - (9)
enter stored value "todayMonthNumber" into "Month"
example - todayMonthNumberTwoDigits - (09)
enter stored value "todayMonthNumberTwoDigits" into "Month"
example - todayMonth - (September)
click stored value "todayMonth"
 example - todayMonthShort - (Sep)
click stored value "todayMonthShort"
 example - todayDayOfMonth - (2)
enter stored value "todayDayOfMonth" into "Month"
example - todayDayOfMonthTwoDigits - (02)
enter stored value "todayDayOfMonthTwoDigits" into "Month"
example - todayDayOfWeek - (Monday)
enter stored value "todayDayOfWeek" into "Day"
example - todayDayOfWeekShort - (Mon)
enter stored value "todayDayOfWeekShort" into "Day"
example - nowHour - (2 or 14)
select stored value "nowHour" from "Start time"
example - nowHourTwoDigits - (02 or 14)
select stored value "nowHourTwoDigits" from "Start time"
example - nowHourAmPm - (2)
select stored value "nowHourAmPm" from "Itinerary"
example - nowHourTwoDigitsAmPm - (02)
select stored value "nowHourTwoDigitsAmPm" from "Itinerary"
example - nowAmPm - (PM)
select stored value "nowAmPm" from "Time of day"
example - nowMinute - (5)
enter stored value "nowMinute" into "Start time"
example - nowMinuteTwoDigits - (05)
enter stored value "nowMinuteTwoDigits" into "Start time"
example - nowSecond - (7)
enter stored value "nowSecond" into "Sec"
example - nowSecondTwoDigits - (07)
enter stored value "nowSecondTwoDigits" into "Sec"
example - nowNanosecond - (355881000)
save string with parameters "email${nowNanosecond}@testrigor-mail.com" as "newEmail"
example - nowDateIso - (2023-09-02)
save stored value "nowDateIso" as "currentDate"
example - nowTimeIso - (02:05:07)
save stored value "nowTimeIso" into "currentTime"
example - nowDateTimeIso - (2023-09-02T02:05:07.165068-07:00[America/Los_Angeles])
save stored value "nowDateTimeIso" as "currentDateTime"
example - nowMillisecondsFrom1970 - (1637061365335)
save stored value "nowMillisecondsFrom1970" as "timeStamp"
example - nowDateTimeRFC1123UTC - (Mon, 16 Sep 2023 16:32:41 GMT)
save stored value "nowDateTimeRFC1123UTC" as "currentDateTime"
example - nowUnixTime - (1637061404)
save stored value "nowUnixTime" as "currentUnixTime"
example - testSuiteParentFolder -
enter stored value "testSuiteParentFolder" into "Field Name"
example - testSuitePath -
enter stored value "testSuitePath" into "Field Name"
example - testSuiteName -
enter stored value "testSuiteName" into "Field Name"
example - testCaseName -
enter stored value "testCaseName" into "Field Name"
example - testCaseExecutionLink -
enter stored value "testCaseExecutionLink" into "Field Name"
example - currentUrl -
save stored value "currentUrl" as "url"
example - browser -
save stored value "browser" as "variable name"
- device -
save stored value "device" as "variable name"
- provider -
save stored value "provider" as "variable name"
- os -
save stored value "os" as "variable name"
- osVersion -
save stored value "osVersion" as "variable name"
- abi -
save stored value "abi" as "variable name"
- appFileName - (only for iOS, android, sandbox)
save stored value "appFileName" as "variable name"
testRigor also supports grabbing values from elements on the screen and saving them into variables for later usage. For instance:
grab value from "some-element" and save it as "my-email"
grab value from input "some-edit" and save it as "my-email"
You can also grab a set of values from a table row or column (the value will be stored as a JSON Array)
grab values from table "some-table" at column "some-column" and save it as "some-column-values"
grab values from table "some-table" at first row and save it as "first-row-values"
You can set variables directly without entering it anywhere. For instance:
generate from regex "[a-z]{10,18}" and save as "actionNotes"
save value "Peter" as "name"
Executing actions in a loop
testRigor has limited support for executing commands until a certain condition is true. For example:
click "Next" until page contains stored value "previously generated id"
It can be used for going through pages in long lists or scrolling down until a certain text is visible or certain button is visible. By default the action will be executed up to 10 times, but you can extend maximum number of times by adding "up to":
click "Next" up to 12 times until page contains strictly button "Place order"
You can do the same thing with rules as well:
Go To The Next Page until page contains stored value "previously generated id"
Do Something up to 12 times until page contains strictly button "Place order"
You can use for wait element appear (or disappear) with a timeout. For example:
wait up to 20 seconds until page contains "non existent button"
Validations
You can make multiple types of validations as described below. Validations are by default are "soft" validations, meaning that the execution of the test case will continue even if the validation failed. You can have "hard" validations by addingand stop test if fails
to the end of the validation.
Validations are supported for multiple things like:
- Finding something on the screen/page
- Validating properties or content of an element/finding element with given properties and content
- Visual validations (colors, matching images to of elements to later versions)
- Downloaded files validations
- Audio validations (Ubuntu only)
- Video validations (desktop web browser testing only)
- Email validations
- SMS validations
- Phone call validations
- Mathematical validations/calculations of formulas (even for data that just looks like financial data like -$30,000.23)
- Built-in API validations
- Chrome Extensions Testing
- AI-based validations where you can use AI to assess if certain statements are true
- Exploratory validations where you can ask AI to look at the screen and find obvious issues
- And many more
For full-page validations, URL and page title validations, we support the following positive and negative assertions:
- contains/doesn't contain (text). For example,
check that page contains "Error"
- contains template/doesn't contain template (text). For example,
check that page contains template "###-###-####"
- contains regex/doesn't contain regex (text). For example,
check that page contains regex "(+\d )?\d{3}-\d{3}-\d{4}"
- return code is. For example,
check that page return code is "404"
- did not change compared to the previous step. For example,
check that page did not change compared to the previous step
- url starts with/url doesn't start with. For example,
check that url starts with "https"
- url contains/url doesn't contain. For example,
check that url contains "testrigor.com"
- url is/url is not. For example,
check that url is "https://testrigor.com/docs/language/"
- url ends with/url doesn't ends with. For example,
check that url is "https://testrigor.com/docs/language/"
- url matches regex/url doesn't match regex. For example,
check that url matches regex "https://testrigor\.com/docs/\w+/"
- title is. For example,
check that page title is "testRigor - Documentation"
- title contains. For example,
check that page title contains "testRigor"
For element-specific validations, we support the following positive and negative assertions:
- contains/doesn't contain (text) - example/example
- is blank/is not blank - example/example
- matches regex/doesn't match regex - example/example
- matches simple template/doesn't match simple template -
check that "section1" matches simple template "$########"
- contains simple template/doesn't contain simple template -
check that "section1" contains simple template "$########"
- contains regex/doesn't contain regex -
check that "section1" contains regex "[A-Z][a-z]+"
- lexicographically before -
check that "section1" lexicographically before "zzzz"
(empty string is not lexicographically before anything) - lexicographically before or the same as -
check that "section1" lexicographically before or the same as "zzzz"
(empty string is not lexicographically before or the same as anything) - lexicographically after -
check that "section1" lexicographically after "AAAA"
(empty string is not lexicographically after anything) - lexicographically after or the same as -
check that "section1" lexicographically after or the same as "AAAA"
(empty string is not lexicographically after or the same as anything) - has value/doesn't have value (for inputs/text areas) - example/example
- is checked/is not checked - example/example
- is disabled/is enabled - example/example
- is visible/is invisible - example/example
- color is - example
- is clickable/is not clickable - example/example
- cursor is - example
- has CSS class - example
- background color is - example
- has property - example
- has options selected (for dropdowns when tagged as
<select>
) - example
check that "element" color is "ffaabb"
check that input "input" has value "value"
check that checkbox "Keep me signed in" is checked and stop test if fails
check that checkbox "Keep me signed in" is not checked
Validate that an input has no value:
check that input "input" has value ""
Validate that element has some property:
check that property "background-color" of "my-div" is equal to "black"
check that property "width" of "my-div" is equal as a number to "310px"
check that property "height" of "my-div" is greater or equal than "170"
For stored values, including: API return value validations, values grabbed from the screen, values extracted from text. We support (positive and negative):
- matches regex/doesn't match regex - example/example
- contains/doesn't contain - example/example
- is equal to/is not equal to - example/example
- is null/is not null - example/example
- is blank/is not blank - example/example
You should use the word
itself
to perform validation on a stored value:call api "https://testrigor.com" and save it as "variableName"
check that stored value "variableName" itself contains "James"
The key to the testing API results is to save it to stored value and then perform validation on stored value itself (with keyword
itself
) as shown above.Both element and stored value validations support:
- is equal as a number to/is not equal as a number to - example/example
- is greater than - example
- is greater or equal than - example
- is less than - example
- is less or equal than - example
You can also enjoy validations that apply to the whole screen/page:
- page contains/page doesn't contain - example/example
- page has regex/page doesn't have regex - example/example
- page's return code is
- title is - example
- title contains - example
- url is/url is not - example/example
- url contains/url doesn't contain - example/example
- url starts with/url doesn't start with - example/example
- url ends with/url doesn't end with - example/example
- url matches regex/url doesn't match regex - example/example
For example, you can combine stored values for validation:
check that url matches regex from the string with parameters "${homePrefix}/product/${type}/[0-9A-Za-z\-]+"
AI-based testing
You can use 2 types of pure-AI-based testing. You have to be cautious since AI can have hallucinations and provide incorrect answer sometimes.- Asking AI specific questions like
check that page "contains a positive message in the chat response" using ai
- example - Getting UI issues from the screen
check page for UI errors
- example
Specific AI validations on the screen
You can set validations to confirm that a statement is true about the screen you are currently on like this:check that page "contains a positive message in the chat response" using ai
See example. Or, alternatively like so:
check that statement is true "page contains TestRigor logo"
Or, you can ask AI to validate statements about specific elements on the screen like so:
check that "element" "contains a positive message" using ai
See example
AI discovers issues itself
You can also ask AI to find issues on a screen like this, you can also specify the limit for the severity for the issues:check page for UI errors
check page for UI errors reporting major errors or higher
check page for UI errors treating errors as major or lower
check page for UI errors reporting major errors or higher treating errors as major or lower
check page for UI errors treating errors as major or lower reporting major errors or higher
See example
Visual Testing
You can verify visual changes on the screen using the following sample steps:
compare screen
compare screenshot to previous version
compare screen to previous version with allowance of "1%" treating error as "minor"
compare screen to previous version treating error as "minor"
*Remember that this verifies that the entire screen looks the same as it did at this step during last successful run.
It is possible to save a sample screenshot in test data and use it as the point of comparison instead of the previously saved screen image:
compare screen to stored value "Saved Screenshot"
Regarding the severity parameter, the allowed values are "minor", "major", "critical", or "blocker". The default value is "critical", which marks the test case as failed. Similarly, there is an option to use the "allowance" clause, a minimum acceptable difference in percentages. Any difference below this value will be ignored. You may need to experiment with this parameter. In any case, you can find actual discrepancy in "Extra info" for a particular step. On mobile devices we ignore the status bar on top. This is so because there is a time stamp that may be different for each screenshot.
Clicking on specific images with visual testing
testRigor also offers visual support for clicking on images by indexing for instances where images or icons do not have a simple way to be referred to. To accomplish this, crop the desired image from a screenshot or file in its full resolution and save it in the Test Data Section. To then refer to it in your case, use the syntax below:
click on the 6th element by image from stored value "logo" with less than "10" % discrepancy
If there is only one image that you want testRigor to recognize, use the following:
click by image from stored value "logo" with less than "10" % discrepancy
You may need to toggle discrepancy percentages in order to allow testRigor to recognize the image.
Working with Tables
You can refer to table cells by the intersection of the row and column by providing value of the first cell in the row and value of the header/top cell in the column. For example, command:
click on table "actions" at row "103" and column "Action"
For the table
# | Id | Name | Actions | Additional Data |
---|---|---|---|---|
Filter by | ||||
101 | york1 | Yorktown | Arrive Cancel | |
102 | spk2 | Spock | Listen to Ignore | |
103 | nyo3 | Nyota | Open channel Promote |
will result in a click on "Open channel".
You can also specify row by saying that row should contain a certain value. This way we will check all values of every row to find the one which matches. For example, for the same table above, and command:
click on table "actions" at row containing "spk2" and column "Actions"
will result in a click on the first action "Listen to". To click on second action "Ignore" you can leverage our context feature:
click on "Ignore" within the context of table "actions" at row containing "spk2" and column "Actions"
You can also work with multiline headers by referring to them wither "header" word:
enter "york1" into first table at the second header row and column "Id"
testRigor supports tables for all kinds of operations including but not limited to: validations (checks), clicks, hover, entering data (enter … into), drag and drop, etc. Examples:
check that table "actions" at row "102" and column "Name" contains "Spock"
check that table "actions" at row "101" and column "Additional Data" has value "Looks like a trap"
enter "This is a trap!" into table "actions" at row "101" and column "Additional Data"
click "Open channel" within the context of second table at row "103" and column "Action"
check that the second table at row containing "Nyota" and column "Action" contains link "Open channel"
click the first button within the context of second table at row containing "Nyota" and column "Action"
Row or column value aggregation
You can check certain aggregations and comparations with the data contained in an entire row or column
Order of values
To check the order of the values contained in a row or column
check that table "actions" at column "Name" has values sorted in ascending order
check that table "actions" at column "Name" has values sorted in descending order
Number of values (count)
To check the value count in a row or column
check that table "actions" at column "Name" the value count is greater than "2"
check that table "actions" at column "Name" the value count is less than "2"
check that table "actions" at column "Name" the value count is equals to "2"
Sum of values
To check the sum of values in a row or column
check that table "actions" at column "#" the sum of values is greater than "100"
check that table "actions" at column "#" the sum of values is less than "500"
check that table "actions" at column "#" the sum of values is equals to "306"
Average of values
To check the average of values in a row or column
check that table "actions" at column "#" the average of values is greater than "100"
check that table "actions" at column "#" the average of values is less than "300"
check that table "actions" at column "#" the average of values is equals to "155"
Validating XLS, XLSX, and CSV File Downloads
You can also use commands related to tables for downloaded XLS, XLSX, and CSV files.click button "Download sample xlsx file"
check that file "file_example_XLSX_50.xlsx" was downloaded
check that file "file_example_XLSX_50.xlsx" at column "Id" the value count is greater than "2"
Conditional execution
testRigor supports conditional execution of commands and rules.
Inline Conditional Command
You can execute a command only if a condition is met, like this:
click "element" if page contains "bla"
enter "macbook" into "search" if "Search in" has value "bla"
In certain cases, certain elements might appear randomly on the screen. For such cases, there is an
if exists
clause. For example:click "element" if exists
enter "bla" into "element" if exists
The above commands will not fail if element is not found and will be skipped silently.
Inline Conditional Rule
You can execute a rule only if a condition is met, like this:
My Rule if page contains "bla"
Purchase Product if "Search in" has value "Products"
Conditional blocks
In addition to inline conditionals, which provide a quick way to execute a command or a rule depending on the result of a validation, we also provide conditional blocks to allow you to test multiple conditions with multiple commands and/or rules.
You can execute multiple commands on one condition. For example:
if page contains "bla" then
click "element"
wait 10 sec
enter "some text" into "some other element"
end
Also if you add an
else
clause you could execute other commands when the condition is not met. For example:if page contains "bla" then
click "element"
wait 10 sec
else
enter "some text" into "search"
wait 10 sec
end
Test as many conditions as you want using the
elseif
clause. For example:if page contains "bla" then
click "element"
wait 10 sec
else if page contains "search" then
enter "some text" into "search"
wait 10 sec
else if page contains "foo" then
enter "foo bar" into "element"
wait 10 sec
click "some button"
else
wait 10 sec
logout
end
In addition to commands you can execute rules inside the conditional blocks. For example:
if page contains "bla" then
check for some words
open the main menu
clear all
end
Conditional blocks are very flexible cause you can mix commands and rules on every condition. For example:
if page contains "Welcome" then
accept all cookies
wait 20 sec
open the main menu
else if page contains "Search" then
click "Search"
enter "Big TV's"
purchase the first item on the list
else
login
end
NOTE: Conditional blocks cannot be nested, we suggest you to use rules to nest complex conditions.
Fail the test on purpose
There is a way to fail the test at will as in the following example:
fail
fail with "error"
API Testing
testRigor supports calling to API's, getting value and saving result as a stored value:
call api <TYPE> "<API_URL>" with headers "a:a" and "b:b" and body "body" and get "JsonPath" and save it as "variableName"
For example:
call api post "http://dummy.restapiexample.com/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body "{\"name\":\"James\",\"salary\":\"123\",\"age\":\"32\"}" and get "$.data.name" and save it as "createdName" and then check that http code is 200
In the example above, testRigor would validate that return http code is 200. Parameters
"JsonPath"
and "variableName"
are both optional. However, if "JsonPath"
is present, "variableName"
will be mandatory. Here is one more example,call api "api" and save it as "variableName"
JsonPath is the way to refer to parts of JSON and described with examples here. If "JsonPath" is not defined in the action, the complete result of the request will be stored in "variableName".
testRigor supports all HTTP actions for call <TYPE>:
- get - example
- post - example
- put - example
- patch - example
- head - example
- delete - example
- options - example
- trace - example
testRigor supports multiple headers separated by "and" like so:
with headers "a:a" and "b:b"
You can pass JSON into the body of POST message, with double quotes characters escaped like so: "\"". You could use tools like this to escape it for you. For example:
and body "{\"name\":\"James\",\"salary\":\"123\",\"age\":\"32\"}"
You can use parameters and multi-line strings for constructing a call:
call api post from the string with parameters "${homePrefix}/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body text starting from next line and ending with [END]
{
"param": "value",
"param2": "value2"
}
[END] and get "$.data.name" and save it as "createdName"
And you can pass parameters:
call api post from the string with parameters "${homePrefix}/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body from the string with parameters text starting from next line and ending with [END]
{
"param": "value",
"param2": "${dynamicValue}"
}
[END] and get "$.data.name" and save it as "createdName"
After the API call is executed, if the result was stored into saved value it could be later tested like so:
check that stored value "createdName" itself contains "James"
Validation of API calls made by the browser (Chrome/Edge only)
Web browsers perform several operations behind the scenes in order to render a page or application, these include multiple requests to retrieve necessary files such as HTML, style sheets, scripts, etc. And also communicating with APIs (application programming interfaces) which provide data in a more dynamic way. testRigor allows you to verify the correct functioning of the mentioned request directly from the steps of the test case through a validation command.
The command accepts several parts for you to filter a particular request or to validate more:
- Request URL: We will find requests to a target URL starting with your input
- Request method: GET, POST, PUT, PATCH, DELETE, etc
- Request headers: filtering among the requests those with headers containing your input
- Request body: filtering requests sending data containing your input
- Response status code: finding a response received with that status code
- Response headers: finding a response received with headers containing the input
testRigor will look for the requests/responses that match the parameters specified on your command, using the priority determined by the following list:
- Filter by request URL matching
- Filter by request method matching
- Filter by request headers matching
- Filter by request body matching
- Filter by response status code
- Filter by response headers
Note: these filters act as a pipeline, so each one will receive the output of the one before.
A validation with all the parameters would look like:
check that api call was made to POST from stored value "myURL" with headers containing "some-header-value" and "\"Content-Type\":\"application/json\"" and text starting from next line and ending with [END]
"Referer":"https://testrigor.com"[END] and the request body containing "some data" and the response code was "200" and response headers containing "resp-header-value"
Simpler ones can be made of course:
check that the browser called api "https://testrigor.com" and response code was "200"
check that api call was made to url "https://testrigor.com" with headers containing from stored value "myHeader"
check that api call was made with response headers "some-value" and from stored value "myHeaderVariable"
Request body types to filter
testRigor allows the user to specify multiple body parameters to filter requests. Each body parameter can be a TEXT, a JSON or a JSON PATH.
Text body parameter
Text is the default body parameter type, so if the user does not specify a type, we will use the text type.
check that api call was made to POST "https://testrigor.com" with body containing "some_data1" and text "some_data2"
JSON body parameter
To use a JSON body parameter, the user must specify the word json before the parameter value. The parameter value must be composed of a key and a value divided by a colon (:) as in the example below.
check that api call was made to POST "https://testrigor.com" with body containing json "key1 : value1" and json "\"key2\":\"value2\""
JSON PATH body parameter
To use a JSON PATH body parameter, the user must specify the word json path before the parameter value. The parameter value must be a valid json path.
check that api call was made to POST "https://testrigor.com" with body containing json path "$.some_path[?(@.value < 10)]"
The user can use more than one type of parameter in the same step.
Mocking API calls
testRigor supports mocking response data (headers, body and/or http status code) for API calls made inside your application.
mock api call <TYPE> "<API_URL>" returning body "<MOCK_BODY>"
For example:
mock api call get "https://dummy.restapiexample.com/api/v1/employees" with headers "a:a" returning body "This is a mock response" with http status code 200
In the example above any GET calls to the endpoint "https://dummy.restapiexample.com/api/v1/employees" with the headers "a:a" will respond with the testRigor mock, with status 200.
Some useful cases are:
- You might want to use mocking APIs if you are using third-party API calls. Those calls can be charged and expensive, so you can mock the responses instead of calling the real service.
- You can test your application individually if servers are unstable or down.
- You can test specific scenarios, for example, if you want to test a scenario where the server returns error.
Uploading files using browser
File upload is supported out of the box. Just use it like the following:
enter stored value "keyName" into input file "fileField"
enter "<FILE_URL>" into file input "fileField"
The most common way to upload files is without specifying the name of the input:
enter stored value "myFile" into input type file
You can also specify the
input type file
field by indexing or relative location:enter stored value "myFile" into 3rd input type file
enter stored value "myFile" into input type file in the context of "Select file"
*Note: In most cases, clicking on the button that opens the file directory is not necessary to upload the file and should be omitted. If this does not work, you can try clicking on the button, entering the file, and then closing the directory with a command appending
using OCR only using the mouse
.Where "fileField" is a file input element (i.e.,
input type="file" ...
), you can upload files up to 10MB into testRigor storage in the "Test Data" section, then use it by name as a stored value. Alternatively you can use your own URL. If you choose to upload from your own URL, the link should be downloadable. Since files must always be uploaded during executions from the testRigor environment (whether stored in test data or called by URL) and cannot be directly uploaded from users' local storage, only in very rare cases is it necessary to include commands in the script to click the button that prompts the file directory or file finder.Multiple files
Multiple files upload can be done like so:
enter from the string with parameters text starting from next line and ending with [END]
${file_var_one}
<FILE_URL>
[END] into file input "myFileInput"
Working with folders
If the input element has a
webkitdirectory
attribute, e.g. - <input type="file" webkitdirectory ...
this element is expecting a folder. If you try uploading a file, we will put it into a temporary folder and enter that value. However, if you want to upload an actual folder, you can zip the folder and use it as an input. In such case, we will unzip the file, preserving directory structure and enter the root path. This is similar to the drag folder action.Note: This does NOT work in headless mode. It also does NOT work with Internet Explorer
Template files
You can also specify files containing comma separated values (CSV) and parameterize those with variables (any stored value) in the form ${nowDateIso}.
The file should be in CSV or TXT format and contain the values and variables in the form:
some value, ${variable1}, another value
${variable2}, fixed value, ${variable3}
To allow templating you need to use the word "template" like the following
enter template stored value "keyName" into input file "fileField"
enter template "<FILE_URL>" into file input "fileField"
Phone calls
testRigor supports making a call through Twilio . There is a section in the application configuration under
Integrations for setting Twilio parameters; it is required for making calls.
example
After the integration is setup you can add custom steps like:
call "+15344297154" and validate it was picked up
make call to "+15344297154" and check it was answered
call to +15344297154
call "+15344297154" and check it was completed
call "+15344297154" from "+15551234567" and verify it is ringing
check that phone call from "+15344297154" is ringing
check that phone call from "+15344297154" was answered
Number to call from is optional and may need to be allocated prior to using. Numbers to call from and to can be from stored values or parameterized strings:
call from stored value "allocatedNumber" to stored value "answerPhoneNumber"
SMS messages (Phone Text)
You can send messages thanks to Twilio integration. There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for sending messages.
Then you can add custom steps like:
sms "+15344297154" with body "this is content" and validate it was sent
example
send sms to "+15344297154" with content "this is content"
send message to "+15344297154" with text "this is content" and check it was delivered
send message from +15551234567 to "+15344297154" with text "this is content" and check it was failed
message from +15551234567 to "+15344297154" with body "this is content" and check it was not delivered
Number from which to send message is optional and may need to be allocated prior to using.
You can also check SMS messages and validate and store their contents:
check that sms from "+12345678901" to "+12345678902" contains "Code" and matches regex "Code\:\d\d\d\d" and save it as "sms"
You can then additionally apply
extract value
command to get just the code:check that sms to "+12345678902" matches regex "Code\:\d\d\d\d" and save it as "sms"
extract value by regex "(?<=Code\:)[0-9]{4}" from "sms" and save it as "confirmationCode"
Number from, number to, and body can be taken from stored values or parameterized strings. Additionally, message body may be a multiline string:
send sms from stored value "allocatedNumber" to stored value "answerPhoneNumber" with content from string with parameters starting from next line and ending with <END>
${answerCode} b
<END>
You can request a temporary phone number from Twilio . There is a section in the application configuration under
Integrations for setting Twilio parameters; it is required for making calls.
After the integration is setup you can add custom steps like:
allocate a temporary number and save it as "newNumber"
This custom step will request a new phone number (Twilio charges will apply) and save it in a variable
"newNumber", which you can use to check for incoming calls or messages. When the test run finished, this
number will be released automatically.
Here is an example of testing a 2FA login with SMS:
click "Sign in"
enter "jacob" into "Username"
enter "jacobs-secure-password" into "Password"
click "Verify me"
check that sms to "+12345678902" is delivered and matches regex "Code\:\d\d\d\d" and save it as "sms"
extract value by regex "(?<=Code\:)[0-9]{4}" from "sms" and save it as "confirmationCode"
enter saved value "confirmationCode" into "code"
click "Continue to Login"
check that page contains text "Welcome, Jacob!"
Login support
testRigor supports login with a single command like so:
login
example
This action identifies and performs the necessary steps required to login to the application automatically. After running successfully the first time, it creates a rule named Autogenerated Login for the application containing the identified steps, that you can override by creating a reusable rule with the same name.
For example, for an application with a simple email and password login, the steps executed and included in the rule would be:
enter stored value "username" into "email"
enter stored value "password" into "password"
click "Log In"
Note that login command relies on having login credentials configured for your application on testRigor.
Email testing
testRigor supports testing with both sending emails as well as receiving and validating emails:
send email to "[email protected]" with subject "Test message" and body "Hi, this is a test, this is just a test message."
check that email from "[email protected]" is delivered
Sending emails also supports sending attachments by referencing a URL of a file you'd like to attach:
send email from "[email protected]" to "[email protected]" with subject "Test message", and body "Hi, this is a test, this is just a test message.", and attachment from saved value "Sample File"
send email from "[email protected]" to "[email protected]" with subject "Test message", and body "Hi, this is a test, this is just a test message.", and attachment "http://online.com/file/name.pdf"
Notes:
- "to" and "from" can be used at the same time. If not specified, "from" address will default to "[email protected]".
- A stored file defined in "Test Data" can be used as an optional attachment. An attachment can also be a link to a file available online without username/password.
- Both "from" and "to" addresses can be filled from stored values (e.g., "to saved value "newEmail" and/or "from saved value "newEmail")
Checking email will automatically open and render it as an HTML page in a desktop browser window:
check that email to "<random-user>@testrigor-mail.com" from "[email protected]" is delivered
check that two emails to "[email protected]" and "Confirm" in subject were delivered
check that one or more emails to "[email protected]" and "Confirm" in subject were delivered
check that email to saved value "newEmail" was received
check that email to saved value "newEmail" from "[email protected]" and "Confirm" in subject was received
check that email to saved value "newEmail" and "<regular expression>" in subject was received
If you need to render the message in the browser of the same mobile device where you test your app or website (if tested on a mobile device), e.g. - when validating a sign up test case, or just want to see how it looks on a small screen, you can mention this in the action like so:
check that email to "<random-user>@testrigor-mail.com" and "Confirm" in subject was received and show in mobile
In some cases you may need to check that email was not sent, here is how to do this:
check that email to "<random-user>@testrigor-mail.com" from "[email protected]" was not delivered
Notes:
-
If the email count is not specified, we assume that only one message is expected. Multiple emails are treated as an error; so in order to avoid receiving the error message, specify the number of emails that should be expected or simply inform the system that more than one can be sent ad done below:
check that one or more emails to saved value "<random-user>@testrigor-mail.com" were received
-
The user should send a message to the following address:
<random-user>@testrigor-mail.com
, where<random-user>
cab be any valid email handle. In order to avoid a conflict it is recommended that the user handle will contain customer name and some random part, e.g. [email protected]. This is important when you are running email tests in multiple browsers and/or on multiple machines at the same time. In this case it is your responsibility to ensure that you are not getting messages, meant for another test case run. -
The user is expected to add some wait time before checking email. This check action will validate delivery, but not wait for the message to arrive. For example:
wait 2 minutes
- Once the action has been executed successfully, the message is deleted from the mailbox and rendered in a new tab. Multiple emails will be marked as error, but each one will be rendered in a separate tab. You can switch between tabs using
switch to tab "1"
,switch to tab "2"
, etc. Note that old content will stay in the tab before the first rendered email message. In most cases it is tab "1", unless you opened more tabs. -
Each email tab is rendered as an html page, so you can use any actions that you would use on an html page. There are a few useful element ids that will help getting specific parts of an email:
- "message-froms" a list of "From" addresses (usually one address)
- "message-subject" subject line
- "message-date" message sent date and time in Pacific Time zone.
- "message-tos" a list of "To" addresses
- "message-ccs" a list of "CC" addresses
- "message-bccs" a list of "BCC" addresses
- "message-reply-to" "Reply To" address
- "message-text" the message text
- "message-attachments" a list of attachments (file names only)
- If you want to check again, you need to re-send the mail. Your custom actions cannot contain multiple "check" actions for the same message.
-
Multiple recipients can be separated with "and":
send email to saved value "newEmail" and saved value "newEmail2" with subject "Test" and body "Hi"
- When checking emails we only take into account those sent after a specific test case has started. This is to avoid a conflict when an email was left unchecked from a previous, unfinished run.
Example of a test for a sign-up flow:
click "Sign up"
generate unique email, then enter into "Email" and save as "generatedEmail"
generate unique name, then enter into "Name" and save as "generatedName"
enter "PasswordSuperSecure" into "Password"
click "Submit"
check that email to stored value "generatedEmail" was delivered
click "Confirm email"
check that page contains "Email was confirmed"
check that page contains expression "Hello, ${generatedName}"
Browser cookies, localStorage, sessionStorage, userAgent
The user can set/get/clear browser cookies like this:
set cookie "cookie value" as "cookie-name"
set cookie from saved value "variableName" as "cookie-name"
get cookie "cookie-name" and save it as "variableName"
clear cookies
set/get/clear browser localStorage and sessionStorage items like so:
set item "item-data" in sessionstorage as "item-name"
get item "item-name" from session storage and save it as "varName"
clear sessionstorage
set item "item-data" in localstorage as "item-name"
get item "item-name" from local storage and save it as "varName"
clear localstorage
and set/unset a custom userAgent value like this:
set user agent to "My User Agent"
unset user agent
Comments support
testRigor supports one-line comments separated by
//
like so:click "my-cryptic-button" // actually clicks "add to cart"
Audio testing
You can test audio by recording it and then comparing it with another recording in the same test or an external file. (Audio testing is currently available for Linux/Ubuntu devices only).
Recording
You have the option to record all the audio that is being reproduced in a tab. The command below in an example:
record audio through 10 seconds after clicking "audio-trigger" and save as "my-recording"
You can also record the audio for a specific audio tag as in the example below:
record audio from "my-audio-tag" through 20 seconds after clicking "audio-trigger" and save as "my-specific-recording"
Comparing
After having a recording that is saved as a variable, you can compare it against another recording. For example:
check that audio from "my-recording" is "70%" similar to "my-specific-recording"
You can also compare it against an external file:
check that audio from "my-recording" is "85%" similar to "https://some-page.com/path-to-file"
The supported file extension for external files is .wav
When comparing we can test positive or negative by using similar or different, for example:
check that audio from "my-recording" is "99%" similar to "my-specific-recording"
or
check that audio from "my-recording" is "1%" different to "my-specific-recording"
Playing
If you need to reproduce audio as it was someone speaking through a microphone you could load the .wav that is going to be reproduced from a remote file.
For tests running in linux, virtual devices are used (microphone/speakers) otherwise we attach to existing
< audio > tags
You can reference a remote file to download
play audio "https://some-page.com/path-to-file"
Or a reference from test data
play audio "test-data-ref"
If more control about when to play the audio and when to stop it is needed, you can use
play audio "sound-to-play-ref" after clicking "start-recording" then click "stop-recording"
Validating
If you need to check that audio is produced (e.g., that a video is playing correctly with sound), you can use the following commands:
check that audio is playing
Negative test is also available:
check that audio is not playing
In both cases, a 10-second audio sample will be recorded, which will be available in the artifacts for download for manual validation if needed.
Database Query Support
You can connect to certain external databases through a JBDC driver and execute commands directly from your tests.
Managing Multiple Connections
It's possible to configure more than one connection. In this case, you can indicate the connection to be used by name:
run sql query "select top 1 UserID, LastName, FirstName from Users;" using connection "connectionName"
run no-sql query "" using connection "anotherConnectionName"
Note: If the connection name is not passed, the application will use the first configured.
Overriding the settings
You can update the connection settings at runtime using the following variables: "connectionName:usernameJDBC", "connectionName:passwordJDBC" and "connectionName:connStringJDBC":
save value "user" as "connectionName:usernameJDBC".
save value "pass12345" as "connectionName:passwordJDBC
save value "jdbc:mysql:host:port" as "connectionName:connStringJDBC"
run sql query "select top 1 UserID, LastName, FirstName from Users;" using connection "connectionName"
SQL Database Support
We have driver support for the most popular databases such as MySQL, PostgreSQL, SQL Server, Snowflake, and Grid Gain.
You can use SQL commands in the test steps to retrieve, insert, update, and delete rows.
Select
The following example will get the first row and save each pair of column-name/value into a stored value with the column name as the key:
run sql query "select top 1 UserID, LastName, FirstName from Users;"
Then you can use the stored values as follows:
enter stored value "FirstName" into "First name"
check that stored value "LastName" itself contains "Doe"
Insert
Executes the defined insert statement, for example:
run sql query "insert into Users(UserID, FirstName, LastName) values (3, 'Jon', 'Doe');"
CSV Files
You can run SQL queries on CSV files.click "download" on the right of "Pokemons.csv"
check that file "Pokemons.csv" was downloaded
run sql query "select Type from Pokemons;" using file "Pokemons.csv"
No-SQL Database Support
We have driver support for MongoDB databases.
You can use MongoDB commands in the test steps to retrieve, insert, update, and delete documents.
Find
The following example will get the first document and save each pair of property-name/value into a stored value with the property name as the key:
run no-sql query "{\"find\": \"Users\", \"limit\": 1, \"projection\": {\"userId\": 1, \"firstName\": 1, \"lastName\": 1}}"
Then you can use the stored values as follows:
enter stored value "FirstName" into "First name"
check that stored value "LastName" itself contains "Doe"
Insert
Executes the defined insert command, for example:
run no-sql query "{\"collection\": \"Users\", \"documents\": [{\"userId\": 3, \"firstName\": \"Jon\", \"lastName\": \"Doe\"}]}"
Chrome Extensions Testing
You can use your extension to test it or test the integration with your/other pages.
A CRX file of the extension is needed for this feature. Upload it in Settings -> Integrations -> Chrome Extensions, and assign a name to it as it will be needed for its use in the commands to open the extension.
To open the extension in a new tab, use the following command:
open extension "myextension"
To open the extension in a pop up or to simply enable the extension, you can use this command:
activate extension "myextension"
Captcha resolution (not included by default)
You can resolve image to text and Google Re-Captcha V2 and V3 directly from your test case steps. Keep in mind this is an extra feature and you will have to contact our sales team to include it in your plan.
Image to text type of captcha
This type of captcha is very common and consists mainly of an image showing a sequence of distorted letters and numbers. To solve it, just indicate in your test the type of captcha as image and point to the element containing the captcha or captcha image. The result of the operation will be saved as a variable called "captchaTest" and you can enter it on the corresponding input.
resolve captcha of type image from "captcha element"
Then, you can use the extracted text as follows:
enter stored value "captchaText" into "Enter captcha text here..."
click "Validate"
Google Re-Captcha type of captcha
Google Re-Captcha comes in many flavors and is usually in the page as a custom script that renders (or not depending on the settings) a button to verify if the user is a human. In this case you don't have to specify an element since we're going to detect the captcha on the page and solve it. If the configuration of the captcha has a callback, we will even call it for you. Also, we're going to store the resulting token into a variable called "captchaToken" so you can enter it on the corresponding input or use it inside a javascript snippet to validate it if such callback is not defined.
resolve captcha of type recaptcha
Then, if a callback was not detected into the configuration of the captcha, you can use the extracted token as follows:
enter stored value "captchaToken" into "g-recaptcha-response"
click "Validate"
Scan QR Code
testRigor supports scan QR Code, getting value and saving result as a stored value.
You can use the Scan QR Code feature to read a QR Code image saved as variable or to read a QR Code image from screen.
Scan QR Code from stored image
scan qr code value from stored value "saved-qr-code" and save as "code"
In the command above we are reading the QR Code image stored as "saved-qr-code" and saving the result in the variable "code"
Scan QR Code from screen
You can specify any element from screen to scan as QR Code.
scan qr code value from "code-container" on the right of "QR Code 2" and save as "code"
In the command above we are reading the QR Code image from element "code-container" that is to the right of "QR Code 2" on the screen and saving the result in the "code" variable
testRigor advanced topics
testRigor provides you full interface to JavaScript with ability to access the full power of testRigor via interface.
JavaScript Support
testRigor allows you to use JavaScript and refer to testRigor's commands like so:
store value "hello" as "var1"
execute JavaScript text starting from next line and ending with [END]
if (testRigor.hasStoredValue("var1")) {
testRigor.execute('enter stored value "var1" into "Message"');
}
testRigor.execute('click "Update"');
testRigor.execute('check that page contains text from stored value "var1"');
[END]
JavaScript supported is fully ECMAScript 5.1 compatible.
testRigor interface
testRigor interface is as following:
interface TestRigor {
/**
* True if variable had been assigned a value.
*/
boolean hasStoredValue(String storedValueName);
/**
* Gets the value of the variable.
*/
String getStoredValue(String storedValueName);
/**
* Sets the value of variable.
*/
String putStoredValue(String storedValueName, String value);
/**
* Extracts the full variable-to-value map.
*/
Map<String, String> getReadOnlyStoredValuesMap();
/**
* Gets tree of elements for the current screen.
*/
UiElement getNodeTree();
/**
* Execute the commands. Can be multiple commands new-line separated.
*/
boolean execute(String commands);
}
Screen/Page structure
The interface UiElement is as following:
interface UiElement {
String getElementName();
String getHtmlNodeName();
double getY();
double getX();
double getHeight();
double getWidth();
UiElement getParent();
String getText();
String getPath();
String getShortPath();
String getFullPath();
String getName();
String getLabel();
String getHint();
String getTitle();
String getClassName();
String getResourceId();
String getOcrText();
String getPlaceholder();
String getAlternative();
String getType();
String getValue();
String getAriaLabel();
String getRole();
boolean isVisible();
boolean isEnabled();
boolean isClickable();
boolean isChecked();
boolean hasProperty(String propertyName);
String getProperty(String propertyName);
}
Above, most of the properties are directly from XML/HTML, except calculated ones:
- x
- y
- height
- width
- visible
- ocrText - only available if OCR is enabled
- shortPath - short version of XPath leveraging ids where possible
- fullPath - direct full XPath from the root
- path - one of the above based on the settings
Note, that id is stored in resourceId attribute
Element interactions
Referencing elements is parsed using ML-based NLP, but can be thought as the following stricture in simplified Pseudo-EBNF:
command [index] [typeOfElement] ["containing"] ["saved value" | "parameterized string"] [elementReference] ["in the context of", elementReference]
Where:
elementReference = \"elementName\", {"or", elementReference} [insideTableReference]
Where:
insideTableReference = "at" ["row", elementReference] ["column", elementReference]
For example:
click on the first button "delete" or "remove" in the context of the first table "visible" at the row containing saved value "generatedId" and column "actions"
Or you can refer elements directly through:
click XPath "/html/body/div[3]"
Or:
click CSS Selector "a[target=_blank]"
Automate testing tables
Selecting a row in a table
testRigor has a very powerful way of dealing with whatever looks like a table for a user.
You can refer to elements of the table by specifying row and column. And to specify a row there is a powerful SQL-like language is available. For example, you can say things like:
click "Delete" inside of table at row containing "my unique id" and column "Actions"
Here we didn't specify the column where the "my unique id" should reside, but rather indicated that it could be in any column at that row.
You can also construct SQL-like expressions containing one or multiple columns with different validations. For example:
check that table with "Country" equal to "United States" and "City" equal to "Saint Petersburg" and "State" equal to "Florida" at column "Status" contains "Flourishing"
Here we used 3 different conditions to identify the row connected with the boolean operator AND. The following boolean operators are supported:
- AND
- OR
- NOT
- Brackets ()
and you can also use brackets. The condition calculation will follow the standard boolean logic. The condition above could also be expressed like so:
check that table with not("Country" not equal to "United States" or "City" not equal to "Saint Petersburg" or "State" not equal to "Florida") at column "Status" contains "Flourishing"
You can learn more about how boolean operators work here.
The exciting thing about testRigor is that these instructions would work regardless of how your table is rendered. It could be rendered as an HTML TABLE tag with TRs and TDs today and completely div-based tomorrow. It doesn't matter, your code will continue to function anyway.
For an example, let's consider the following table:
# | Id | Name | Ship | Status | Actions |
---|---|---|---|---|---|
1 | york1 | Yorktown | Enterprise | Alright | Arrive Cancel |
2 | spk2 | Spock | Enterprise | Alright | Listen to Ignore |
3 | nyo3 | Nyota | Enterprise | Alright | Open channel Promote |
4 | spk2 | Spock | Kelvin | Alright | Listen to Ignore |
5 | nyo3 | Nyota | Kelvin | Alright | Open channel Promote |
In the example above, rows have no unique meaningful identifiers. We only have row numbers which might be generated automatically to show the index of a row. We can't rely on them consistently because they will change the next time the page refreshes. How are we going to on a button with Spock and Kelvin? Here is how:
click on the first button in the context of table with "Name" equal to "Spock" and "Ship" equals to "Kelvin" at column "Actions"
Calculating aggregates
You can calculate and work with aggregate values on columns or specify a condition that would validate against all values in a column. Here are some examples:
Order of values
To check the order of the values contained in a row or column
check that table "actions" at column "Name" has values sorted in ascending order
check that table "actions" at column "Name" has values sorted in descending order
Number of values (count)
To check the value count in a row or column
check that table "actions" at column "Name" the value count is greather than "3"
check that table "actions" at third column the value count is less than "20"
check that table "actions" at column "Name" the value count is equals to "5"
check that table "actions" at third column the value count is equals to stored value "expectedValueCount"
Sum of values
To check the sum of values in a row or column
check that table "actions" at column "#" the sum of values is greater than "10"
check that table "actions" at first column the sum of values is less than "50"
check that table "actions" at column "#" the sum of values is equals to "15"
check that table "actions" at first column the sum of values is less than stored value "expectedValueSummatory"
Average of values
To check the average of values in a row or column
check that table "actions" at column "#" the average of values is greater than "1"
check that table "actions" at first column the average of values is less than "6"
check that table "actions" at column "#" the average of values is equals to "3"
check that table "actions" at first column the average of values is equals to stored value "expectedValueAvg"
JsonPath Overview
testRigor uses JsonPath as a way to help you to refer to elements of returned JSON. JsonPath expressions always refer to a JSON structure in the same way as XPath expression are used in combination with an XML document. The "root member object" in JsonPath is always referred to as
$
regardless if it is an object or array. JsonPath expressions can use the dot-notation$.store.book[0].title
or the bracket-notation
$['store']['book'][0]['title']
Operators
Operator | Description |
---|---|
$ |
The root element to query. This starts all path expressions. |
@ |
The current node being processed by a filter predicate. |
* |
Wildcard. Available anywhere a name or numeric are required. |
.. |
Deep scan. Available anywhere a name is required. |
.<name> |
Dot-notated child |
['<name>' (, '<name>')] |
Bracket-notated child or children |
[<number> (, <number>)] |
Array index or indexes |
[start:end] |
Array slice operator |
[?(<expression>)] |
Filter expression. Expression must evaluate to a boolean value. |
Functions
Functions can be invoked at the tail end of a path - the input to a function is the output of the path expression. The function output is dictated by the function itself.
Function | Description | Output |
---|---|---|
min() |
Provides the min value of an array of numbers | Double |
max() |
Provides the max value of an array of numbers | Double |
avg() |
Provides the average value of an array of numbers | Double |
stddev() |
Provides the standard deviation value of an array of numbers | Double |
length() |
Provides the length of an array | Integer |
sum() |
Provides the sum value of an array of numbers | Double |
Filter Operators
Filters are logical expressions used to filter arrays. A typical filter would be
[?(@.age > 18)]
where @
represents the current item being processed. More complex filters can be created with logical operators &&
and ||
. String literals must be enclosed by single or double quotes ( [?(@.color == 'blue')]
or [?(@.color == "blue")]
).Operator | Description |
---|---|
== |
left is equal to right (note that 1 is not equal to '1') |
!= |
left is not equal to right |
< |
left is less than right |
<= |
left is less or equal to right |
> |
left is greater than right |
>= |
left is greater than or equal to right |
=~ |
left matches regular expression [?(@.name =~ /foo.*?/i)] |
in |
left exists in right [?(@.size in ['S', 'M'])] |
nin |
left does not exists in right |
subsetof |
left is a subset of right [?(@.sizes subsetof ['S', 'M', 'L'])] |
anyof |
left has an intersection with right [?(@.sizes anyof ['M', 'L'])] |
noneof |
left has no intersection with right [?(@.sizes noneof ['M', 'L'])] |
size |
size of left (array or string) should match right |
empty |
left (array or string) should be empty |
Path Examples
Given the JSON
{
"store": {
"book": [
{
"category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{
"category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99
},
{
"category": "fiction",
"author": "Herman Melville",
"title": "Moby Dick",
"isbn": "0-553-21311-3",
"price": 8.99
},
{
"category": "fiction",
"author": "J. R. R. Tolkien",
"title": "The Lord of the Rings",
"isbn": "0-395-19395-8",
"price": 22.99
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
},
"expensive": 10
}
The following results will be returned:
JsonPath | Result |
---|---|
$.store.book[*].author |
The authors of all books |
$..author |
All authors |
$.store.* |
All things, both books and bicycles |
$.store..price |
The price of everything |
$..book[2] |
The third book |
$..book[-2] |
The second to last book |
$..book[0,1] |
The first two books |
$..book[:2] |
All books from index 0 (inclusive) until index 2 (exclusive) |
$..book[1:2] |
All books from index 1 (inclusive) until index 2 (exclusive) |
$..book[-2:] |
Last two books |
$..book[2:] |
Book number two from tail |
$..book[?(@.isbn)] |
All books with an ISBN number |
$.store.book[?(@.price < 10)] |
All books in store cheaper than 10 |
$..book[?(@.price <= $['expensive'])] |
All books in store that are not "expensive" |
$..book[?(@.author =~ /.*REES/i)] |
All books matching regex (ignore case) |
$..* |
Give me every thing |
$..book.length() |
The number of books |
Regex Random String Generation Support
In addition to a simple way to generate random data, testRigor also supports a way to use Regular Expressions (Regex) for more complex scenarios. For instance, you can generate a random tag like this:
generate by regex "<(a|button)>data<\/\1>", then enter into "Tag data" and save as "generatedTag"
which generates
<a>data</a>
or <button>data</button>
. Or you might wantgenerate by regex "https://(www\.)?([a-z][a-z0-9-]{2,30}[a-z0-9]\.){1,5}(com|edu|co\.uk|info|io)", then enter into "URL" and save as "generatedUrl"
which generates a unique URL in the format of "https://www.abc.de.com".
The rule of thumb to use ReGex is to avoid it as much as possible since it is very hard to read for anyone who is not familiar with it. Always try simple template first. One of the use cases where you might be forced to use ReGex is if what you generate must have a variable length.
testRigor supports both generation of random strings based on Regex as well as validating and extracting data.
Regex Random Data Generation Cookbook
For simple scenarios like generating emails, phone numbers, credit card numbers, etc. please see simple template. The full command will look like this:
generate by regex "<YOUR_REGEX>", then enter into "Email" and save as "newEmail"
Regex | Use case |
---|---|
[A-Z][a-z]{2,50} |
Random word starting from upper case like "Hello" |
[a-z]{2,50} |
Random word all lower case like "hi" |
[1-2][0-9]{0,8} |
Random number |
((1[0-2])|0[1-9])-((0[1-9])|([1-2][0-8]))-((20[0-2][0-9])|(19[0-9]{2})) |
Dates in mm-dd-yyyy format |
((20[0-2][0-9])|(19[0-9]{2}))-((1[0-2])|0[1-9])-((0[1-9])|([1-2][0-8])) |
Dates in yyyy-mm-dd format |
((1[0-2])|(0[0-9])):([0-5][0-9]) [AP]M |
Time in 11:34 AM AM/PM format |
((2[0-3])|([0-1][0-9])):([0-5][0-9]) |
Time in 14:25 24hr format |
(www\.)?([a-z][a-z0-9-]{2,30}[a-z0-9]\.){1,5}[a-z]{2,3} |
Domain name |
https://(www\.)?([a-z][a-z0-9-]{2,30}[a-z0-9]\.){1,5}[a-z]{2,3} |
URL without path |
https?://(www\.)?([a-z][a-z0-9-]{2,30}[a-z0-9]\.){1,5}[a-z]{2,3}(/[a-z0-9-]{2,10}){0,5}/?(#[a-z0-9])? |
Full random URL |
((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?) |
IP v4 |
((\+[0-9][0-9]? [0-9]{3})|(\(0[0-9]{3}\))) [0-9]{3} [0-9]{4} |
International phone numbers in a standard E.123 format |
Regex Random Generation Syntax Support
testRigor provides full support for Java-like regular expressions with POSIX character classes and full multibyte Unicode support for international characters. A simple example is:
generate by regex "[A-Z][a-z]{2,50}", then enter into "Name" and save as "generatedName"
Construct | Description | Example |
---|---|---|
A text |
All regular text is copied as is. This is not changeable part. In the example on the right Lakshmi will be posted as is. | generate by regex "Lakshmi", then enter into "Name" and save as "name" |
| |
Select one option from another. testRigor will randomly choose from the options available. In the example on the right, it will use either Lakshmi or Meera | generate by regex "Lakshmi|Meera", then enter into "Name" and save as "name" |
() |
Grouping parts. In case if something needs to be executed on the whole rater than the last symbol. Lakshmi+ will result in "Lakshmiiiiii", where as (Lakshmi )+ will result in "Lakshmi Laskhmi Lakshmi ..." |
generate by regex "(Lakshmi )+", then enter into "Name" and save as "name" |
? |
Add or do not add the part randomly. The example would result in "Lakshmi" about 50% or the time and empty string other 50% | generate by regex "(Lakshmi)?", then enter into "Name" and save as "name" |
+ |
Repeat the part at least one and at most 100 times. The example on the right will have at least one Lakshmi but can also have up to 99 more. | generate by regex "(Lakshmi )+", then enter into "Name" and save as "name" |
* |
Repeat the part at least zero and at most 100 times. The example on the right could produce an empty string or up to 100 "Lakshmi" strings. | generate by regex "(Lakshmi )*", then enter into "Name" and save as "name" |
{N} |
Repeat the part precisely N times. In example on the right "Lakshmi " will be repeated twice like this: "Lakshmi Lakshmi ". | generate by regex "(Lakshmi ){2}", then enter into "Name" and save as "name" |
{N,M} |
Repeat the part between N and M times. In example on the right "Lakshmi " can be repeated either once or twice. | generate by regex "(Lakshmi ){1,2}", then enter into "Name" and save as "name" |
[abc] |
Select just one of the characters specified. In the example only a, b, or c will be selected randomly. | generate by regex "[abc]", then enter into "Name" and save as "name" |
[a-c] |
Select just one of the characters in the range specified. In the example only a, b, or c will be selected randomly. | generate by regex "[a-c]", then enter into "Name" and save as "name" |
[^abc] |
If the first symbol in square brackets is ^ , then testRigor will use visible symbols except the ones provided. In the example, it could be any character like 'A', '7', 'd', '&', etc. This reversal applies universally and can be used with ranges as well, like [^a-c] , which in this case, will have the same effect. |
generate by regex "[^abc]", then enter into "Name" and save as "name" |
\s |
One character of any whitespace like space or tab. | generate by regex "\s", then enter into "Name" and save as "name" |
\w |
One character of lower and upper case Latin letters, digits and '_'. Equivalent to [a-zA-Z0-9_] . |
generate by regex "\w", then enter into "Name" and save as "name" |
\d |
One digit character. Equivalent to [0-9] . |
generate by regex "\d", then enter into "Name" and save as "name" |
. |
Any character. testRigor will generate characters that can be typed on a US keyboard randomly. | generate by regex "\d", then enter into "Name" and save as "name" |
\ |
Escapes the following special character. For example, if you'd like to include '[' and ']' into the set of characters you use, you could specify[\[\]] |
generate by regex "[\[\]]", then enter into "Name" and save as "name" |
\n |
Just one new line character. | generate by regex "\n", then enter into "Name" and save as "name" |
\t |
Just one tab character. | generate by regex "\t", then enter into "Name" and save as "name" |
\N |
A reference to the previously generated Nth group. For example, <(a|button)>data<\/\1> will generate the opening and closing of the same tag ("a" or "button" in this case). |
generate by regex "<(a|button)>data<\/\1>", then enter into "Code" and save as "code" |
Advanced Topics
When we were creating the Regex generator we wanted the syntax to be as close as it is technically possible to Java version of Regex (Pattern). You might find a lot of additional quirks and features to be similar to Java's version. However, the parser was built from scratch and had to be generation-compatible, so there are some differences. For instance, we included support for almost all POSIX character classes. However, we emulated how Java works with backreferences and group numbering. Also, we tried to build parser in a way that would produce the most readable and understandable error messages we could do.
Question | Answer |
---|---|
How does testRigor generates random? | testRigor generates random numbers with pseudo-random sequence seeded with nanoseconds. |
Is there a bound for unbounded operations like * ? |
Yes, default bound is 100 |
What is I need more than 100 symbols generated? | You can use range, and specify any number as upper bound. testRigor will repeat up to your specified number or 10,000, whichever is smaller. |
Does testRigor support intersections like [a&&[b]] ? |
Yes for validation/search. Not yet for random data generation. |
Does testRigor support multicharacter Unicode symbols? | Yes, naturally out of the box. You can specify 😀{3} to get "😀😀😀" |
Does testRigor support non-capturing groups? | Yes for validation/search. For generation positive non-capturing groups like (?<=group) will generate data, but negative groups like (?!group) will be ignored. |
Does testRigor support flags like (?i) ? |
Yes for validation/search. Not yet for random data generation. |
More simple use cases like phone numbers, passport numbers, SSNs, emails, etc. see simple template.
Regex for validations and search
You can use Regex for search/validation in testRigor. testRigor will use Java 11's Regex Pattern to complile/process the Regex in this case. See the full supported syntax here. Examples of usages are:
check that page has regex "Current time: [0-2][0-9]:[0-5][0-9]"
check that page doesn't have regex "fail|crash|500"
check that URL matches regex "https://mycompany\.com/list/item/[a-z0-9]{2,30}"
check that text below "Current time" matches regex "[0-2][0-9]:[0-5][0-9]"
grab value of regex "[0-2][0-9]:[0-5][0-9]" and save it as "currentTime"
How to use simple templates to generate unique data
testRigor provides as a way to help you to easily generate unique data. For instance, you can generate unique data like phone numbers like this:
generate from template "###-555-####", then enter into "Phone" and save as "generatedPhone"
Which would generate a unique phone number like
752-555-0912
. To do that testRigor uses 4 different symbols to indicate types of data you want to generate: #$%*
.Unique data generating symbols
Symbol | Description |
---|---|
# |
Generates a random number in range 0-9. |
$ |
Generates a random lower case letter in range a-z. |
% |
Generates a random upper case letter in range A-Z. |
* |
Generates a random alphanumeric symbol, either number, or lower or upper case letter (0-9, a-z, A-Z). |
\ |
If the following symbol is one of the above or another \ then this \ symbol is removed and the following symbol is returned verbatim instead of generating a random symbol. |
Examples
Keep in mind, that testRigor has already a built-in words to generate unique email or unique name. You do not need to provide a template for them. Those would work like so:
generate unique email, then enter into "Email" and save as "newEmail" generate unique name, then enter into "Name" and save as "generatedName"
Example | Description |
---|---|
generate unique email, then enter into "Email" and save as "newEmail" |
Generates a random email in testrigor-mail.com domain. |
generate unique name, then enter into "Name" and save as "generatedName" |
Generates a random name. |
generate from template "$******************************@testrigor-mail.com", then enter into "Email" and save as "newEmail" |
Generates a random email in a custom domain. |
generate from template "###-###-####", then enter into "Phone" and save as "generatedPhone" |
Generates a random phone number. |
generate from template "811-###-####", then enter into "Phone" and save as "generatedPhone" |
Generates a random phone number in a 811 area code. |
generate from template "000-##-####", then enter into "SSN" and save as "generatedSsn" |
Generates a random SSN number. |
generate from template "4###-####-####-####", then enter into "card" and save as "generatedCard" |
Generates a random VISA credit card. |
generate from template "https://$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$.info", then enter into "url" and save as "generatedUrl" |
Generates a random URL. |
generate from template "%******************************", then enter into "data" and save as "generatedData" |
Generates a random alphanumeric data. |
generate from template by string with escaped parameters "${nowDateTimeIso}-**********", then enter into "Data" and save as "generatedData" |
Generates a random alphanumeric data starting with date-time in ISO format. |
generate from template "%$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$ $$$$$$$$$$$$$$$$$$$$$$$$$$$$$$.", then enter into "description" and save as "generatedDescription" |
Generates a random text description. |
More complex use cases in some cases might require ReGex and can be found here.
What is Boolean Logic
Boolean Logic in the context of testRigor is a way to combine boolean conditions like
"Status" contains "Success"
. You can use three main operators to join these conditions:- AND
- OR
- NOT
- Brackets ()
Here are some examples:
AND operator
AND operator helps to join conditions which both must be true for the result to be true. For example,
"Status" contains "Success" and "City" is equal to "Paris"
will only be true when the selected row contains "Success" in column "Status" and has string "Paris" in column "City" at the same time.OR operator
OR operator helps to find a row containing one or both conditions. For example,
"Status" contains "Success" or "City" is equal to "Paris"
will only be true when the selected row either contains "Success" in column "Status" or has string "Paris" in column "City" or both.NOT operator
NOT operator helps calculate the negative of the condition. For example,
not "Status" contains "Success"
will only be true when the selected row does not contain "Success" in column "Status".Brackets ()
Brackets () help to group conditions together. For example,
("Status" contains "Success" or "Status" contains "In progress") and "City" is equal to "Paris"
will only be true when the selected row contains "Success" or "In progress" in column "Status" and has string "Paris" in column "City" at the same time. In this example "Status" column must contain either "Success" or "In progress", and column "City" must contains "Paris".