testRigor English-language support documentation

All of the below apply to both directly creating tests as well as creating subroutines called "rules". Rule is a way to give a better name to a sequence of steps.

You can create a rule named "go to product page" and then use it in your test like this:

go to product page
check page contains "Add to cart"

By default, you should be able to write basic steps like below in plain English, but data and variables are in double-quotes. The list below is just an example of how you can express it, we actually support many other ways to do it.

All references must be in double quotes, which can be escaped by backslash. There are several ways to refer to elements (for checks, clicks, entering data, etc.):

1. References. check that "peter" color is "ffaabb" will search:
   1. Nested text
   2. Placeholder
   3. Value
   4. data-tid/data-testid/data-test-id
   5. Id/resource-id/data-id
   6. Name
   7. aria-label
   8. CSS Class
   9. Label from ML classification
   10. Hint/Title/Tooltip
   11. Alt/Src
   12. For inputs/edits/dropdowns/selects/etc. will also search corresponding label
2. Generic indexes. For example: check that second "peter" color is "ffaabb"
3. Types. For example, check that input "peter" color is "ffaabb" will fail id "peter" is not an input/edit. Supported types:
   1. text
   2. label
   3. button
   4. link
   5. input (or "edit" or "field")
   6. dropdown (or "select")
   7. checkbox (or "switch")
   8. radiobutton
   9. file input (or "input file" or "edit file" or "input type file"): specifically an input of type file
4. Typed indexes. For example, check that second input "peter" color is "ffaabb" will find the second input/edit related to "peter".
5. Stored value for controls. For example, check that stored value "peter" color is "ffaabb" will resolve stored value "peter" and use that resolved value to find the control.
6. Multiple references. For example: check that "peter" or "john" color is "ffaabb" will find the first control related to either "peter" or "john".
7. Image class. For example: click on "cart". testRigor classifies images on the screen and if there is a button which looks like a shopping cart it will click on it. Complete list of all button image categories is here.
8. Stored value for data. For example: check that "peter" contains stored value "actionNote" will find the second input/edit related to "peter".

Multiple references support.

click "checkout" or "submit"

click "checkout" or "submit" if exists

Selecting elements in reference to other elements

click on button "Delete" below "Actions"

1. to the left of
2. to the right of
3. above
4. below
5. on the right top of
6. on the left top of
7. on the right bottom of
8. on the left bottom of
9. near

click on button "Delete" below "Section Name" to the right of "label"

1. completely - default overlap with the target area is at least 30%
2. with at least "10" percent overlap - similar to above but overlap can be specified
3. roughly - element is just anywhere below the cut line for "below", to the right of for "to the right of"

enter "Peter" into roughly below "Section"

enter "Peter" into element with at least "1" percent overlap on the right of "Description"

enter "Peter" into below "Section" on the right of "Description"

enter "Peter" into roughly below "Section" completely on the right of "Description"

Selecting elements in the context of other elements

click on "Delete" within the context of table "actions" at row containing "id1" and column "Actions"

click on "Delete" within the context of "sectionTwo" below "Actions" and to the right of "rowName"

Selecting elements using reference to other elements and in the context of other elements

1. Context is processed first, narrowing what to consider to what is inside that element
2. References to other elements is processed next, only considering what was delimited by the context

click on "Button" below "Title" in the context of "sectionOne"

click on "Button" below "Title" and to the right of "leftHeader" in the context of "sectionTwo"

click on "Button" below "Subtitle" and to the right of "leftHeader" in the context of "sectionOne" below "Title" and above "Second Title"

Specifying type of an element

click on strictly button "Delete"

Case sensitivity and exact matches

click on exactly "Delete"

click on strictly button exactly "Delete"

Fine-tuning access

click deepest element "Delete"

Specifying position on screen

click on "Delete" with offset "20,10"

click in the middle of the screen

1. in the middle of the screen
2. in the top quarter of the screen
3. in the second top quarter of the screen
4. in the bottom quarter of the screen
5. in the second bottom quarter of the screen
6. in the top third of the screen
7. in the bottom third of the screen
8. in the left quarter of the screen
9. in the second left quarter of the screen
10. in the right quarter of the screen
11. in the second right quarter of the screen
12. in the left third of the screen
13. in the right third of the screen
14. in the left side of the screen
15. in the right side of the screen
16. in the top of the screen
17. in the bottom of the screen
18. on the left edge of the screen
19. on the right edge of the screen
20. on the top edge of the screen
21. on the bottom edge of the screen
22. in the left top corner of the screen
23. in the right top corner of the screen
24. in the left bottom corner of the screen
25. in the right bottom corner of the screen

generate by regex "[a-z]{10,18}", then enter into "Notes" and save as "actionNotes"
validate that stored value "actionNotes" color is "ffaabb"
check that "peter" contains stored value "actionNote"
click on stored value "actionNotes"
check that page contains stored value from "actionNotes"

enter stored value "username" into "username_field"
enter stored value "password" into "password_field"

enter from the string with parameters "${homePrefix}/my/path" into "urlPath"

check that expression "${a} + ${b}" itself is equal as a number to "42"

check that page contains expression "new Date().setDate(today.getDate()-30).getFullYear()"

1. username
2. password
3. homeDomain
4. homeFile
5. homePrefix
6. todayYear
7. todayYearShort
8. todayMonthNumber
9. todayMonthNumberTwoDigits
10. todayMonth
11. todayMonthShort
12. todayDayOfMonth
13. todayDayOfMonthTwoDigits
14. todayDayOfWeek
15. todayDayOfWeekShort
16. nowHour
17. nowHourTwoDigits
18. nowMinute
19. nowMinuteTwoDigits
20. nowSecond
21. nowSecondTwoDigits
22. nowNanosecond
23. nowDateIso
24. nowTimeIso
25. nowDateTimeIso
26. nowMillisecondsFrom1970
27. nowUnixTime

grab value from "some-element" and save it as "my-email"
grab value from input "some-edit" and save it as "my-email"

generate by regex "[a-z]{10,18}" and save as "actionNotes"
save value "Peter" as "name"

click "Next" until page contains stored value "previously generated id"

click "Next" up to 12 times until page contains strictly button "Place order"

For UI validations for elements we support (positive and negative):

1. contains/doesn't contain (text) check that "my_div" contains "error"
2. is blank/is not blank
3. matches regex/doesn't match regex
4. has value/doesn't have value (for inputs/text areas)
5. is checked/is not checked
6. is disabled/is enabled
7. is visible/is invisible
8. color is
9. is clickable/is not clickable
10. cursor is
11. has CSS class
12. background color is
13. has property
14. has options selected (for selects)

check that "element" color is "ffaabb"
check that input "input" has value "value"
check that checkbox "Keep me signed in" is checked
check that checkbox "Keep me signed in" is not checked

Validate that an input has no value:

check that input "input" has value ""

Validate that element has some CSS property:

check that property "background-color" of "my-div" is equal to "black"

For stored values, including: API return value validations, values grabbed from the screen, values extracted from text. We support (positive and negative):

1. matches regex/doesn't match regex
2. contains/doesn't contain
3. is equal to/is not equal to
4. is null/is not null
5. is blank/is not blank

You should use the word itself to perform validation on a stored value:

call api "https://testrigor.com" and save it as "variableName"
check that stored value "variableName" itself contains "James"

The key to the testing API results is to save it to stored value and then perform validation on stored value itself (with keyword itself) as shown above.

Both element and stored value validations support:

1. is equal as a number to/is not equal as a number to
2. is greater than
3. is greater or equal than
4. is less than
5. is less or equal than

You can also enjoy validations that apply to the whole screen/page:

1. page contains/page doesn't contain check that page contains "error message"
2. page has regex/page doesn't have regex
3. page's return code is
4. title is
5. title contains
6. url is/url is not
7. url contains/url doesn't contain
8. url starts with/url doesn't start with
9. url ends with/url doesn't end with
10. url matches regex/url doesn't match regex

For example, you can combine stored values for validation:

check that url matches regex from the string with parameters "${homePrefix}/product/${type}/[0-9A-Za-z\-]+"

You can refer to table cells by the intersection of the row and column by providing value of the first cell in the row and value of the header/top cell in the column. For example command:

click on table "actions" at row "103" and column "Action"

For the table

will result in a click on "Open channel".

You can also specify row by saying that row should contain a certain value. This way we will check all values of every row to find the one which matches. For example, for the same table above, and command:

click on table "actions" at row containing "bbbbb2" and column "Actions"

will result in a click on the first action "Listen to". To click on second action "Ignore" you can leverage our context feature:

click on "Ignore" within the context of table "actions" at row containing "bbbbb2" and column "Actions"

You can also work with multiline headers by referring to them wither "header" word:

enter "aaaaa1" into first table at the second header row and column "Id"

testRigor supports tables for all kinds of operations including but not limited to: validations (checks), clicks, hover, entering data (enter … into), drag and drop, etc. Examples:

check that table "actions" at row "102" and column "Name" contains "Spock"
check that table "actions" at row "101" and column "Additional Data" has value "Looks like a trap"
enter "This is a trap!" into table "actions" at row "101" and column "Additional Data"
click "Open channel" within the context of second table at row "103" and column "Action"
check that the second table at row containing "Nyota" and column "Action" contains link "Open channel"
click the first button within the context of second table at row containing "Nyota" and column "Action"

In certain cases, certain elements might appear randomly on the screen. For such cases, there is an if exists clause. For example:

click "element" if exists
enter "bla" into "element" if exists

the above commands will not fail if element is not found and will be skipped silently.

There is a way to fail at will as in the following example:

fail
fail with "error"

testRigor supports calling to API’s, getting value and saving result as a stored value:

call api <TYPE> "<API_URL>" with headers "a:a" and "b:b" and body "body" and get "JsonPath" and save it as "variableName"

For example:

call api post "http://dummy.restapiexample.com/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body "{\"name\":\"James\",\"salary\":\"123\",\"age\":\"32\"}" and get "$.data.name" and save it as "createdName"  and then check that http code is 200

In the example above testRigor would validate that return http code is 200. Parameters "JsonPath" and "variableName" are both optional. However, if "JsonPath" is present, "variableName" will be mandatory. Here is one more example:

call api "api" and save it as "variableName"

JsonPath is the way to refer to parts of JSON and described with examples here. If "JsonPath" is not defined in the action, the complete result of the request will be stored in "variableName".

testRigor supports all HTTP actions for call <TYPE>:

1. get
2. post
3. put
4. patch
5. head
6. delete
7. options
8. trace

testRigor supports multiple headers separated by "and" like so: with headers "a:a" and "b:b"

You can pass JSON into the body of POST message, with double quotes characters escaped like so: "\"". You could use tools like this to escape it for you. For example: and body "{\"name\":\"James\",\"salary\":\"123\",\"age\":\"32\"}"

You can use parameters and multi-line strings for constructing a call:

call api post from the string with parameters "${homePrefix}/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body text starting from next line and ending with <END>
{
  "param": "value",
  "param2": "value2"
}
<END> and get "$.data.name" and save it as "createdName"

And you can pass parameters:

call api post from the string with parameters "${homePrefix}/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body from the string with parameters text starting from next line and ending with <END>
{
  "param": "value",
  "param2": "${dynamicValue}"
}
<END> and get "$.data.name" and save it as "createdName"

After the API call call is executed, if the result was stored into saved value it could be later tested like so:

check that stored value "createdName" itself contains "James"

File upload is supported out of the box. Just use it like the following:

enter stored value "keyName" into input file "fileField"
enter "<FILE_URL>" into file input "fileField"

Where "fileField" is a file input element, i.e.: <input type="file" ... You can upload the file into testrigor store in "Test Data" section, then use it by name as stored value. Alternatively you can use your own URL. If you choose to upload from your own URL, the link should be downloable. In any case the file is uploaded from within testRigor environment.

Multiple files

Multiple files upload can be done like so:

enter from the string with parameters text starting from next line and ending with <END>
${file_var_one}
<FILE_URL>
<END> into file input "myFileInput"

testRigor supports making a call through Twilio . There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for making calls.

After the integration is setup you can add custom steps like:

call "+15344297154" and validate it was picked up
make call to "+15344297154" and check it was answered
call to +15344297154
call "+15344297154" and check it was completed
call "+15344297154" and verify it is ringing
check that phone call from "+15344297154" is ringing
check that phone call from "+15344297154" was answered

You can request a temporary phone number from Twilio . There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for making calls.

After the integration is setup you can add custom steps like:

allocate a temporary number and save it as "newNumber"

This custom step will request a new phone number (Twilio charges will apply) and save it in a variable "newNumber", which you can use to check for incoming calls or messages. When the test run finished, this number will be released automatically.

You can send messages thanks to Twilio integration. There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for sending messages.

Then you can add custom steps like:

sms "+15344297154" with body "this is content" and validate it was sent
send sms to "+15344297154" with content "this is content"
send message to "+15344297154" with text "this is content" and check it was delivered
send message to "+15344297154" with text "this is content" and check it was failed
message to "+15344297154" with body "this is content" and check it was not delivered

You can also check SMS messages and validate and store their contents:

check that sms from "+12345678901" to "+12345678902" contains "Code" and matches regex "Code\:\d\d\d\d" and save it as "sms"

You can then additionally apply extract value command to get just the code:

check that sms to "+12345678902" matches regex "Code\:\d\d\d\d" and save it as "sms"
extract value by regex "(?<=Code\:)[0-9]{4}" from "sms" and save it as "confirmationCode"

testRigor supports login with a single command like so:

login

This action identifies and performs the necessary steps required to login to the application automatically. After running successfully the first time, It creates a rule named Autogenerated Login for the application containing the identified steps, that you can override by creating a subroutine/rule with the same name.

For example: for an application with a simple email and password login, the steps executed and included in the rule would be:

enter stored value "username" into "email"
enter stored value "password" into "password"
click "Log In"

Note that login command relies on having login credentials configured for your application on testRigor.

testRigor supports testing with both sending emails as well as receiving and validating emails:

send email to "user@customer.com" with subject "Test message" and  body "Hi, this is a test, this is just a test message."

check that email from "user@customer.com" is delivered

Sending emails also supports sending attachments by referencing a URL of a file you’d like to attach:

send email from "sender@customer.com" to "recipient@customer.com" with  subject "Test message", and  body "Hi, this is a test, this is just a test message.", and attachment from saved value "Sample File"
send email from "sender@customer.com" to "recipient@customer.com" with  subject "Test message", and  body "Hi, this is a test, this is just a test message.", and attachment "http://online.com/file/name.pdf"

Notes:

1. "to" and "from" can be used at the same time. If not specified, "from" address will default to "noreply@testrigor-mail.com"
2. A stored file, defined in "Test Data" can be used as an optional attachment. An attachment can also be a link to a file available online without username/password
3. Both "from" and "to" addresses can be filled from stored values, e.g. - "to saved value "newEmail"" and/or "from saved value "newEmail""

Checking email will automatically open and render it as an HTML page:

check that email to "random-user@testrigor-mail.com" from "user@customer.com" is delivered
check that two emails to "user@customer.com" and "Confirm" in subject were delivered
check that one or more emails to "user@customer.com" and "Confirm" in subject were delivered
check that email to saved value "newEmail" was received
check that email to saved value "newEmail" from "admin@customer.com" and "Confirm" in subject was received
check that email to saved value "newEmail" and "<regular expression>" in subject was received

In some cases you may need to check that email was not sent, here is how to do this:

check that email to "random-user@testrigor-mail.com" from "user@customer.com" was not delivered

Notes:

1. If email count is not specified, we assume that only one message is expected. Multiple emails is treated as an error.
2. The user should send a message to the following address: <random-user>@testrigor-mail.com, where <random-user> can be any string. In order to avoid overlap it is recommended that the user handle will contain customer name and some random part, e.g. hfynerifj@testrigor-mail.com.
3. The user is expected to add some wait time before checking email. This check action will validate delivery, but not wait for the message to arrive. e.g.:

   wait 2 minutes

4. Once the action executed successfully, the message is deleted from the mailbox and rendered in a new tab. Multiple emails will be marked as error, but each one will be rendered in a separate tab. You can switch between tabs using switch to tab "1", switch to tab "2", etc. Note that old content will stay in tab, before the first rendered email message. In most cases it is tab "1", unless you opened more tabs.
5. Each email tab is basically an html page and you can use any actions that you would use on an html page. There are a few useful element ids that will help getting specific email parts:
   • "message-froms" a list of "From" addresses (usually one address)
   • "message-subject" subject line
   • "message-date" message sent date and time in Pacific Time zone.
   • "message-tos" a list of "To" addresses
   • "message-ccs" a list of "CC" addresses
   • "message-bccs" a list of "BCC" addresses
   • "message-reply-to" "Reply To" address
   • "message-text" the message text
   • "message-attachments" a list of attachments (file names only)
6. If you want to check again, you need to re-send the mail. Your script cannot contain multiple "check" actions for the same message.
7. Multiple recipients can be separated with "and":

   send email to saved value "newEmail" and saved value "newEmail2" with subject "Test" and  body "Hi"

8. All messages will be deleted after 24 hours, whether checked or not.

The user can set/get/clear browser cookies like this:

set cookie "cookie value" as "cookie-name"
set cookie from saved value "variableName" as "cookie-name"
get cookie "cookie-name" and save it as "variableName"
clear cookies

Also the user can set/get/clear browser localStorage and sessionStorage items like so:

set item "item-data" in sessionstorage as "item-name"
get item "item-name" from session storage and save it as "varName"
clear sessionstorage
set item "item-data" in localstorage as "item-name"
get item "item-name" from local storage and save it as "varName"
clear localstorage

testRigor supports one-line comments separated by // like so:

click "my-cryptic-button" // actually clicks "add to cart"

You can test audio by recording it and then comparing it with another recording in the same test or an external file.

Recording

You have the option to record all the audio that are being reproduced in a tab, for example

record audio through 10 seconds after clicking "audio-trigger" and save as "my-recording"

or you can record the audio for a specific audio tag like

record audio from "my-audio-tag" through 20 seconds after clicking "audio-trigger" and save as "my-specific-recording"

Comparing

After having a recording saved as a variable you can compare it against another recording, for example

check that audio from "my-recording" is "70%" similar to "my-specific-recording"

or you can compare it against an external file

check that audio from "my-recording" is "85%" similar to "https://some-page.com/path-to-file"

The supported file extension for external files is .wav

When comparing we can test positive or negative by using similar or different, for example

check that audio from "my-recording" is "99%" similar to "my-specific-recording"

or

check that audio from "my-recording" is "1%" different to "my-specific-recording"