testRigor English-language support documentation
The goal of testRigor is to allow you to write your tests in your way of saying it in plain English.
Rules (Subroutines)
All of the below apply to both directly creating tests as well as creating subroutines called "rules". Rule is a way to give a better name to a sequence of steps.
You can create a rule named "go to product page" and then use it in your test like this:
go to product page
check page contains "Add to cart"
By default, you should be able to write basic steps like below in plain English, but data and variables are in double-quotes. The list below is just an example of how you can express it, we actually support many other ways to do it.
Table of commands
Action | Options | Example |
---|---|---|
click | double, right, times, in a context of | or: or: or: or: or:
|
generate | RegEx | Generate email: Generate multi-line: Generate and type: Generate and save only:
|
enter | enter, tab, escape, backspace | and Note: enter supports selects/dropdowns/checkboxes/etc For checkbox you can do: For enter, tab, backspace: For multi-line inputs: For selects: The previously described applies also for entering without a reference to an input: or: Note: You will need to handle focusing on the input where to enter |
check | page | and
|
check | element | and You can also check for is disabled, is clickable, is enabled Check stored value itself: Check that some element looks the same: Check that SMS is received: Check that file is downloaded and check its content:
|
long press |
| |
hover |
| |
switch tab | or Note: If an option is a number, it will be used as a tab position number. If it is not a number, it will be used as a tab window name. | |
go back |
| |
go forward |
| |
reload |
| |
reset to home |
| |
drag | element, file | Or file: Or draw on a canvas: If you need to drag folder with files, then you need to zip it and upload in Test Data section and use it like so: Note: this action does NOT work in headless mode, also it does NOT work with Internet Explorer |
scroll | direction | Or:
|
swipe | direction |
|
wait | time | Note: 2 min max |
call api | URL | Or with more parameters:
|
grab value | element | or, to grab a value from UserName:53vhhsw1fi@testrigor-mail.com or:
|
extract value by regex | element | for example, to extract a value of username from variable var1 containing UserName:53vhhsw1fi@testrigor-mail.com you could use:
|
save value |
| |
open url | URL | Note: The domain must be whitelisted! |
send email | Here is how to send a simple email: To send an email with an attachment, you can upload the file into testrigor store in "Test Data" section, then use it by name as stored value. Alternatively you can use your own URL. If you choose to attach a file from your own URL, the link should be downloable.
| |
check email | Or You can use both - sender and recepient Also it is possible to filter messages by subject Note: this command will render the email as if it is a web page. The to address here is not the same as "To" address in the email. check that email to will validate that email is delivered to one of the email recepients (To, Cc, Bcc). If there are more than one recepient, several commands may check for every recepient's address. | |
reply to email | Or Note: this command will render the received email (not the response) as if it is a web page | |
call | phone number | Or, simply:
|
sms | phone number | Or
|
set geo location |
| |
start browser / start device | name | Starts a new browser with its own new session. Start and switch: For device:
|
switch to browser / switch to device | name | For device: Get back to the first browser:
|
compare | type of element | Compares current elements on the current screen with the ones appearing the previous time the screen was shown. The following sentence compares that element looks like during last successful run pixelwise: The sentence must contain an image definition, such as image of text "pressed" , image of 3rd "peter" , etc. Optionally it may contain an allowance value in percent. If missing, then it is assumed to be zero. If element size changed, centered intersection of the element is compared. Pixels are compared using mean squared error. |
login | Complex action. Identifies and performs the necessary steps to login. |
Referencing Locations and Elements
All references must be in double quotes, which can be escaped by backslash. There are several ways to refer to elements (for checks, clicks, entering data, etc.):
- References.
check that "peter" color is "ffaabb"
will search:- Nested text
- Placeholder
- Value
- data-tid/data-testid/data-test-id
- Id/resource-id/data-id
- Name
- aria-label
- CSS Class
- Label from ML classification
- Hint/Title/Tooltip
- Alt/Src
- For inputs/edits/dropdowns/selects/etc. will also search corresponding label
- Generic indexes. For example:
check that second "peter" color is "ffaabb"
- Types. For example,
check that input "peter" color is "ffaabb"
will fail id "peter" is not an input/edit. Supported types:- text
- label
- button
- link
- input (or "edit" or "field")
- dropdown (or "select")
- checkbox (or "switch")
- radiobutton
- Typed indexes. For example,
check that second input "peter" color is "ffaabb"
will find the second input/edit related to "peter". - Stored value for controls. For example,
check that stored value "peter" color is "ffaabb"
will resolve stored value "peter" and use that resolved value to find the control. - Multiple references. For example:
check that "peter" or "john" color is "ffaabb"
will find the first control related to either "peter" or "john". - Image class. For example:
click on "cart"
. testRigor classifies images on the screen and if there is a button which looks like a shopping cart it will click on it. Complete list of all button image categories is here. - Stored value for data. For example:
check that "peter" contains stored value "actionNote"
will find the second input/edit related to "peter".
Multiple references support.
In certain cases, certain elements might be expected to have different names but mean the same thing. For such cases, there is an "or" clause. For example:click "checkout" or "submit"
And it can be combined with "if exists" to not to fail is the element doesn't exist:click "checkout" or "submit" if exists
Selecting elements in reference to other elements
You can select elements based on it's position in relation to some other element. For example, you can refer to elements in certain sections like:click on button "Delete" below "Actions"
Supported relative locations are:to the left of
to the right of
above
below
on the right top of
on the left top of
on the right bottom of
on the left bottom of
near
click on button "Delete" below "Section Name" to the right of "label"
On top of it you can specify how the relative position should be used:completely
- default overlap with the target area is at least 30%with at least "10" percent overlap
- similar to above but overlap can be specifiedroughly
- element is just anywhere below the cut line for "below", to the right of for "to the right of"
enter "Peter" into roughly below "Section"
enter "Peter" into element with at least "1" percent overlap on the right of "Description"
Note, that the default behavior is "completely". However, if there are 2 anchors specified then the first anchor would have default "roughly" and the second one "completely". I.e.enter "Peter" into below "Section" on the right of "Description"
is equivalent to:enter "Peter" into roughly below "Section" completely on the right of "Description"
Selecting elements in the context of other elements
You can select elements within the context of other elements. For example, you can pinpoint row at the table and ask to click a button in the context of that row:click on "Delete" within the context of table "actions" at row containing "id1" and column "Actions"
Specifying type of an element
You can force the system to only deal with certain type of elements. The following code will not click on any text that is not a button or link:click on strictly button "Delete"
Case sensitivity and exact matches
You can choose to deal with only "exactly" matches, i.e. case sensitive full strings match, or just say "case sensitive" to make matching case sensitive:click on exactly "Delete"
It can be combined with the strict type selection like so:click on strictly button exactly "Delete"
Fine-tuning access
In some case the target might be too wide, in this case you can force it to go deeper:click deepest element "Delete"
It will issue click on the deepest element in the element tree that has the text.Specifying position on screen
It is possible but highly discouraged (for stability reasons) to use offsets to click on specific part of an element:click on "Delete" with offset "20,10"
offset is calculated from top left corner of the element, horizontal coordinate first. It is also possible to specify some generic positions like:click in the middle of the screen
The supported positions are:in the middle of the screen
in the top quarter of the screen
in the second top quarter of the screen
in the bottom quarter of the screen
in the second bottom quarter of the screen
in the top third of the screen
in the bottom third of the screen
in the left quarter of the screen
in the second left quarter of the screen
in the right quarter of the screen
in the second right quarter of the screen
in the left third of the screen
in the right third of the screen
in the left side of the screen
in the right side of the screen
in the top of the screen
in the bottom of the screen
on the left edge of the screen
on the right edge of the screen
on the top edge of the screen
on the bottom edge of the screen
in the left top corner of the screen
in the right top corner of the screen
in the left bottom corner of the screen
in the right bottom corner of the screen
Saved values (variables) support
testRigor supports generating random values based on RegEx , saving them and using them later in the test. For example:generate by regex "[a-z]{10,18}", then enter into "Notes" and save as "actionNotes"
validate that stored value "actionNotes" color is "ffaabb"
check that "peter" contains stored value "actionNote"
click on stored value "actionNotes"
check that page contains stored value from "actionNotes"
There are 2 special stored values: "username"
and "password"
which come from Application-specific credentials settings for your test suite. With it you can do:enter stored value "username" into "username_field"
enter stored value "password" into "password_field"
these stored credentials are also used by login
command. Also, you can use stored variables as parameters in most commands:enter from the string with parameters "${homePrefix}/my/path" into "urlPath"
And calculate expressions like this:check that expression "${a} + ${b}" itself is equal as a number to "42"
. testRigor supports ECMAScript 5.1 compatible expressions. For example, you can calculate a year from 30 days ago is present on the screen:check that page contains expression "new Date().setDate(today.getDate()-30).getFullYear()"
testRigor supports the following pre-defined saved values:- username
- password
- homeDomain
- homeFile
- homePrefix
- todayYear
- todayYearShort
- todayMonthNumber
- todayMonthNumberTwoDigits
- todayMonth
- todayMonthShort
- todayDayOfMonth
- todayDayOfMonthTwoDigits
- todayDayOfWeek
- todayDayOfWeekShort
- nowHour
- nowHourTwoDigits
- nowMinute
- nowMinuteTwoDigits
- nowSecond
- nowSecondTwoDigits
grab value from "some-element" and save it as "my-email"
grab value from input "some-edit" and save it as "my-email"
You can set variables directly without entering it anywhere. For instance:generate by regex "[a-z]{10,18}" and save as "actionNotes"
save value "Peter" as "name"
Executing actions until condition (loops) support
testRigor has limited support for executing commands until certain condition is true. For example:click "Next" until page contains stored value "previously generated id"
It can be used for going through pages in long lists or scrolling down until certain text is visible or certain button is visible. By default the action will be executed up to 10 times, but you can extend maximum number of times by adding "up to":click "Next" up to 12 times until page contains strictly button "Place order"
Validations
For UI validations for elements we support (positive and negative):
- contains/doesn't contain (text)
check that "my_div" contains "error"
- is blank/is not blank
- matches regex/doesn't match regex
- has value/doesn't have value (for inputs/text areas)
- is checked/is not checked
- is disabled/is enabled
- is visible/is invisible
- color is
- is clickable/is not clickable
- cursor is
- has CSS class
- background color is
- has options selected (for selects)
check that "element" color is "ffaabb"
check that input "input" has value "value"
check that checkbox "Keep me signed in" is checked
check that checkbox "Keep me signed in" is not checked
Validate that an input has no value:
check that input "input" has value ""
For stored values, including: API return value validations, values grabbed from the screen, values extracted from text. We support (positive and negative):
- matches regex/doesn't match regex
- contains/doesn't contain
- is equal to/is not equal to
- is null/is not null
- is blank/is not blank
You should use the word itself
to perform validation on a stored value:
call api "https://testrigor.com" and save it as "variableName"
check that stored value "variableName" itself contains "James"
The key to the testing API results is to save it to stored value and then perform validation on stored value itself (with keyword itself
) as shown above.
Both element and stored value validations support:
- is equal as a number to/is not equal as a number to
- is greater than
- is greater or equal than
- is less than
- is less or equal than
You can also enjoy validations that apply to the whole screen/page:
- page contains/page doesn't contain
check that page contains "error message"
- page has regex/page doesn't have regex
- page's return code is
- title is
- title contains
- url is/url is not
- url contains/url doesn't contain
- url starts with/url doesn't start with
- url ends with/url doesn't end with
- url matches regex/url doesn't match regex
For example, you can combine stored values for validation:
check that url matches regex from the string with parameters "${homePrefix}/product/${type}/[0-9A-Za-z\-]+"
Working with Tables
You can refer to table cells by the intersection of the row and column by providing value of the first cell in the row and value of the header/top cell in the column. For example command:
click on table "actions" at row "103" and column "Action"
For the table
# | Id | Name | Actions | Additional Data |
---|---|---|---|---|
Filter by | ||||
101 | aaaaa1 | Yorktown | Arrive Cancel | |
102 | bbbbb2 | Spock | Listen to Ignore | |
103 | ccccc3 | Nyota | Open channel Promote |
will result in a click on "Open channel".
You can also specify row by saying that row should contain a certain value. This way we will check all values of every row to find the one which matches. For example, for the same table above, and command:
click on table "actions" at row containing "bbbbb2" and column "Actions"
will result in a click on the first action "Listen to". To click on second action "Ignore" you can leverage our context feature:
click on "Ignore" within the context of table "actions" at row containing "bbbbb2" and column "Actions"
You can also work with multiline headers by referring to them wither "header" word:
enter "aaaaa1" into first table at the second header row and column "Id"
testRigor supports tables for all kinds of operations including but not limited to: validations (checks), clicks, hover, entering data (enter … into), drag and drop, etc. Examples:
check that table "actions" at row "102" and column "Name" contains "Spock"
check that table "actions" at row "101" and column "Additional Data" has value "Looks like a trap"
enter "This is a trap!" into table "actions" at row "101" and column "Additional Data"
click "Open channel" within the context of second table at row "103" and column "Action"
check that the second table at row containing "Nyota" and column "Action" contains link "Open channel"
click the first button within the context of second table at row containing "Nyota" and column "Action"
Conditional execution
In certain cases, certain elements might appear randomly on the screen. For such cases, there is an if exists
clause. For example:
click "element" if exists
enter "bla" into "element" if exists
the above commands will not fail if element is not found and will be skipped silently.
There is a way to fail at will as in the following example:
fail
fail with "error"
API Testing
testRigor supports calling to API’s, getting value and saving result as a stored value:
call api <TYPE> "<API_URL>" with headers "a:a" and "b:b" and body "body" and get "JsonPath" and save it as "variableName"
For example:
call api post "http://dummy.restapiexample.com/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body "{\"name\":\"James\",\"salary\":\"123\",\"age\":\"32\"}" and get "$.data.name" and save it as "createdName" and then check that http code is 200
In the example above testRigor would validate that return http code is 200. Parameters "JsonPath"
and "variableName"
are both optional. However, if "JsonPath"
is present, "variableName"
will be mandatory. Here is one more example:
call api "api" and save it as "variableName"
JsonPath is the way to refer to parts of JSON and described with examples here. If "JsonPath" is not defined in the action, the complete result of the request will be stored in "variableName".
testRigor supports all HTTP actions for call <TYPE>:
- get
- post
- put
- patch
- head
- delete
- options
- trace
testRigor supports multiple headers separated by "and" like so: with headers "a:a" and "b:b"
You can pass JSON into the body of POST message, with double quotes characters escaped like so: "\"". You could use tools like this to escape it for you. For example: and body "{\"name\":\"James\",\"salary\":\"123\",\"age\":\"32\"}"
You can use parameters and multi-line strings for constructing a call:
call api post from the string with parameters "${homePrefix}/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body text starting from next line and ending with <END>
{
"param": "value",
"param2": "${dynamicValue}"
}
<END> into "Notes"
After the API call call is executed, if the result was stored into saved value it could be later tested like so:
check that stored value "createdName" itself contains "James"
Uploading files
File upload is supported out of the box. Just use it like the following:
enter stored value "keyName" into "fileField"
enter "<FILE_URL>" into "fileField"
Where "fileField" is a file input element, i.e.: <input type="file" ...
You can upload the file into testrigor store in "Test Data" section, then use it by name as stored value. Alternatively you can use your own URL. If you choose to upload from your own URL, the link should be downloable. In any case the file is uploaded from within testRigor environment.
Phone calls
testRigor supports making a call through Twilio . There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for making calls.
After the integration is setup you can add custom steps like:
call "+15344297154" and validate it was picked up
make call to "+15344297154" and check it was answered
call to +15344297154
call "+15344297154" and check it was completed
call "+15344297154" and verify it is ringing
check that phone call from "+15344297154" is ringing
check that phone call from "+15344297154" was answered
Requesting temporary incoming phone number
You can request a temporary phone number from Twilio . There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for making calls.
After the integration is setup you can add custom steps like:
allocate a temporary number and save it as "newNumber"
This custom step will request a new phone number (Twilio charges will apply) and save it in a variable "newNumber", which you can use to check for incoming calls or messages. When the test run finished, this number will be released automatically.
SMS messages (Phone Text)
You can send messages thanks to Twilio integration. There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for sending messages.
Then you can add custom steps like:
sms "+15344297154" with body "this is content" and validate it was sent
send sms to "+15344297154" with content "this is content"
send message to "+15344297154" with text "this is content" and check it was delivered
send message to "+15344297154" with text "this is content" and check it was failed
message to "+15344297154" with body "this is content" and check it was not delivered
You can also check SMS messages and validate and store their contents:
check that sms from "+12345678901" to "+12345678902" contains "Code" and matches regex "Code\:\d\d\d\d" and save it as "sms"
You can then additionally apply extract value
command to get just the code:
check that sms to "+12345678902" matches regex "Code\:\d\d\d\d" and save it as "sms"
extract value by regex "(?<=Code\:)[0-9]{4}" from "sms" and save it as "confirmationCode"
Login support
testRigor supports login with a single command like so:
login
This action identifies and performs the necessary steps required to login to the application automatically. After running successfully the first time, It creates a rule named Autogenerated Login for the application containing the identified steps, that you can override by creating a subroutine/rule with the same name.
For example: for an application with a simple email and password login, the steps executed and included in the rule would be:
enter stored value "username" into "email"
enter stored value "password" into "password"
click "Log In"
Note that login command relies on having login credentials configured for your application on testRigor.
Email testing
testRigor supports testing with both sending emails as well as receiving and validating emails:
send email to "user@customer.com" with subject "Test message" and body "Hi, this is a test, this is just a test message."
check that email from "user@customer.com" is delivered
Sending emails also supports sending attachments by referencing a URL of a file you’d like to attach:
send email from "sender@customer.com" to "recipient@customer.com" with subject "Test message", and body "Hi, this is a test, this is just a test message.", and attachment from saved value "Sample File"
send email from "sender@customer.com" to "recipient@customer.com" with subject "Test message", and body "Hi, this is a test, this is just a test message.", and attachment "http://online.com/file/name.pdf"
Notes:
- "to" and "from" can be used at the same time. If not specified, "from" address will default to "noreply@testrigor-mail.com"
- A stored file, defined in "Test Data" can be used as an optional attachment. An attachment can also be a link to a file available online without username/password
- Both "from" and "to" addresses can be filled from stored values, e.g. - "to saved value "newEmail"" and/or "from saved value "newEmail""
Checking email will automatically open and render it as an HTML page:
check that email to "random-user@testrigor-mail.com" from "user@customer.com" is delivered
check that email to saved value "newEmail" was received
check that email to saved value "newEmail" from "admin@customer.com" and "Confirm" in subject was received
check that email to saved value "newEmail" and "<regular expression>" in subject was received
Notes:
- The user should send a message to the following address:
<random-user>@testrigor-mail.com
, where<random-user>
can be any string. In order to avoid overlap it is recommended that the user handle will contain customer name and some random part, e.g. hfynerifj@testrigor-mail.com. - The user is expected to add some wait time before checking email. This check action will validate delivery, but not wait for the message to arrive. e.g.:
wait 2 minutes
- Once the action executed successfully, the message is deleted from the mailbox and rendered in a new tab. Multiple emails will be marked as error, but each one will be rendered in a separate tab. You can switch between tabs using switch to tab "1", switch to tab "2", etc. Note that old content will stay in tab, before the first rendered email message. In most cases it is tab "1", unless you opened more tabs.
- Each email tab is basically an html page and you can use any actions that you would use on an html page. There are a few useful element ids that will help getting specific email parts:
- "message-froms" a list of "From" addresses (usually one address)
- "message-subject" subject line
- "message-date" message sent date and time in Pacific Time zone.
- "message-tos" a list of "To" addresses
- "message-ccs" a list of "CC" addresses
- "message-bccs" a list of "BCC" addresses
- "message-reply-to" "Reply To" address
- "message-text" the message text
- "message-attachments" a list of attachments (file names only)
- If you want to check again, you need to re-send the mail. Your script cannot contain multiple "check" actions for the same message.
- Multiple recipients can be separated with "and":
send email to saved value "newEmail" and saved value "newEmail2" with subject "Test" and body "Hi"
- All messages will be deleted after 24 hours, whether checked or not.
Audio testing
You can test audio by recording it and then comparing it with another recording in the same test or an external file.
Recording
You have the option to record all the audio that are being reproduced in a tab, for example
record audio through 10 seconds after clicking "audio-trigger" and save as "my-recording"
or you can record the audio for a specific audio tag like
record audio from "my-audio-tag" through 20 seconds after clicking "audio-trigger" and save as "my-specific-recording"
Comparing
After having a recording saved as a variable you can compare it against another recording, for example
check that audio from "my-recording" is "70%" similar to "my-specific-recording"
or you can compare it against an external file
check that audio from "my-recording" is "85%" similar to "https://some-page.com/path-to-file"
The supported file extension for external files is .wav
When comparing we can test positive or negative by using similar or different, for example
check that audio from "my-recording" is "99%" similar to "my-specific-recording"
or
check that audio from "my-recording" is "1%" different to "my-specific-recording"
Comments support
testRigor supports one-line comments separated by
//
like so: