Basic Commands Reusable Rules (Subroutines) Referencing locations Using variables Loops Validations Visual Testing Working with Tables Conditional execution API Testing Uploading files Phone calls SMS messages (Phone Text) Login support Email testing Browser cookies, localStorage, sessionStorage, userAgent Comments Audio Testing SQL Testing Chrome Extension Testing
testRigor English-language support documentation
The goal of testRigor is to allow you to write your tests in your way of saying it in plain English. It is extendable by allowing you to support your own phrases. And you might want to get familiar with test automation best practices and, more importantly testRigor Best Practices. You can click on elements withclick "Submit"
, enter data with enter "Peter" into "First Name"
and validate with check that page contains "Welcome, Peter!"
.
Table of Basic Commands
We support a vast variety of commands. The list below contains many examples of how to express them.
Action | Options | Example |
---|---|---|
click | double, right, middle, long, times, in a context of, using the mouse, using javascript, without scroll, using OCR, using OCR only |
or:
or:
or:
or:
or:
or:
or:
or:
For web testing you can specify the way we click on an element as follows:
or:
Note: In headless mode this option will be ignored as the only method available now is javascript
or:
Note: You can use without scroll to prevent testRigor's default behavior (scroll to the element before the click).
or:
Note: This sample uses an image stored with the name "Logo" in "Test Data". The image is used to identify the location of the click. This type of click is useful when e.g. you need to click on an image map. Default discrepancy is 20 % |
generate | Simple Template, unique email, unique name, RegEx, Google authenticator code |
Generate a unique email in testrigor-mail.com domain:
Generate a unique name:
Generate a unique phone number:
Generate a unique string of letters/numbers:
Generate unique the date and time with unique parameters:
Generate an email address in a custom domain:
Generate multi-line:
Generate and type:
Generate and save only:
Generate Google authenticator code for 2 steps validation login. It needs the user QRCode saved in a stored value to generate the code:
|
enter | enter, tab, escape, delete, backspace, ctrl+a, arrow right, arrow left, arrow up, arrow down, go, search, send, next, previous, done |
Note: enter supports selects/dropdowns/checkboxes/etc.
For checkboxes, you can do the following:
For enter, tab, escape, delete, backspace, ctrl+a, arrows or Android-specific go, search, send, next, previous, done:
For multi-line inputs:
For selects or dropdowns:
Also, it's possible to select an option by position in selects or dropdowns:
|
type |
For typing free text into input fields |
You may type text without referencing an input field if the previous step has already placed the cursor within the input field you wish to use:
For multi-line inputs:
|
copy and paste | text |
Copy selected text (highlighted by either double clicking or dragging the mouse) to the clipboard:
Copy text from string (non-headless only):
example
Copy text from saved value (non-headless only):
example
In order to paste copied text click on the input field and use one of 2 commands:
or simply
|
check | page |
Check page content by string:
or
Check page content (including the OCR recognized texts) by string:
Check page content using OCR recognized texts only:
Check that page did not change after some action:
Note: this action compares page images pixel to pixel
Check that page doesn't have a fourth button:
|
check | element |
and
You can also validate whether an input/checkbox/button/etc. is disabled, clickable, or enabled
Check stored value itself:
Check that an element looks the same as during last successful run:
Check that the entire screen looks the same as during last successful run:
Check that SMS is received:
Check that file is downloaded and check its content:
You can also reuse the downloaded file or multiple files to upload them later in the test. For this you need the following action:
The variable "DownloadedFiles" will contain all files downloaded after the last check for downloaded files. Then to upload the file on some other page, just use the following action
You can also extract text from a downloaded file and save it as a variable.
Check that a container contains a nth element:
Check that an element is changing. This is useful to check that some element that contains animation or video is updating (desktop web browser testing only):
You can also use a negative check:
|
long press |
|
|
hover |
|
|
open new tab |
|
|
switch tab |
or
Note: In order to easily keep track of tab numbers, testRigor's default setting is to begin each test case in a new browser window. Therefore, tab "1" is always the tab the test case starts in, and tab "2", "3" and so on are the new tabs in the order that they are opened throughout the test case run. For pop-ups, instead of a tab number, you will need to use the popup window name or title. |
|
close tab |
|
|
go back |
|
|
go forward |
|
|
reload |
|
|
reset to home |
|
|
restart app |
Restarts application without clearing data.
Note: This action is for mobile devices only. |
|
drag | element, file |
Or file:
Or draw on a canvas:
Drag mouse to multiple points without releasing the click:
If you need to drag a folder with files, you need to zip it and upload in Test Data section and use it like so:
Note: This action does NOT work in headless mode. It also does NOT work with Internet Explorer |
scroll | down, up, left, right |
Or:
Partial scroll:
You can also scroll directly to a specific part of the page:
Or:
Using the mouse:
For pages with several scroll areas, you can also scroll up or down on a specific portion of text using the mouse wheel:
If the target text is very far from the starting point of the scroll, you can focus scrolls on the area of the text instead of the text itself by using mouse wheel action in a loop:
Important Note: The scroll until page contains method works best in Visible first mode. Visible first prioritizes only what is visible in the viewport/screenshot. Batched mode opens testRigor's visibility to anything that is loaded on the page whether it is visible to the end user or not. (To toggle this setting, you can find the dropdown in Settings->Speed optimizations->Performance->Getting visibility of elements approach) |
swipe | down, up, left, right |
Or:
|
wait | time |
Note: 2 min max |
call api | URL |
Or with more parameters:
|
mock api call | URL, returning (headers, body or http status code) |
Or with more parameters:
|
grab value | Simple Template, RegEx, element |
or, to grab a value from
UserName:[email protected]
or:
or, to grab a value using simple template:
or:
You can also grab multiple values from a table row or column (the value will be stored as a JSON Array)
|
extract value by regex | element |
for example, to extract a value of username from variable
var1 containing UserName:[email protected] you could use:
|
save value |
|
|
store clipboard |
Clipboard current value can be saved into a variable for later usage:
|
|
open url | URL |
Note: The domain must be whitelisted! |
grab url |
You can grab the browser's current URL and save it into a variable:
|
|
send email |
Here is how to send a simple email:
To send an email with an attachment, you can upload the file into testRigor store in "Test Data" section, then use it by name as stored value. Alternatively you can use your own URL. If you choose to attach a file from your own URL, the link should be downloable.
|
|
check email |
Or
By default testRigor renders emails in a desktop browser. If you want to render it in the mobile device (available for Android and iOS), you need to add the following suffix:
and show in mobile E.g.:
You can use both - sender and recipient
By default we assume that the user expects one email message. Multiple emails will trigger an error message. It is possible to customize this behavior by specifying exactly how many emails are expected or just say "one or more".
Also it is possible to filter messages by subject
Notes:
|
|
reply to email |
Or
Note: this command will render the received email (not the response) as if it is a web page |
|
call | phone number |
Or, simply:
|
sms | phone number |
Or
|
set geo location (GPS Coordinates) |
|
|
start browser / start device | name |
Starts a new browser with its own new session.
Start and switch:
For device:
|
switch to browser / switch to device | name |
For device:
Get back to the first browser:
|
switch context to native / switch context to browser |
Switch the context of the following actions to the entire mobile device (as for application testing) or start the mobile browser and use only its content (as for web testing). |
|
compare | type of element |
Compares current elements on the current screen with the ones appearing the previous time the screen was shown.
The following sentence compares element on the current test run screen with the one appearing on the reference screen or saved as a file pixelwise. If the element appears to be different, both baseline and current (failed) run element images will be saved in the artifacts for manual investigation.
The sentence must contain an image definition, such as
image of text "pressed" , image of 3rd "peter" , etc. Optionally it may contain an allowance value in percent. If missing, then it is assumed to be zero. If element size changed, centered intersection of the element is compared. Pixels are compared using mean squared error.Silimarly you can make a screenshot of an image, save it, and then use as a reference like so:
|
login |
Complex action. Identifies and performs the necessary steps to login. |
|
crawl sitemap / go through sitemap |
testRigor supports XML and Text sitemap formats.
|
|
execute JavaScript in the browser |
testRigor supports execution of vanilla Javascript in the browser.
|
|
accept alert/prompt | prompt value |
Browser alert/prompt can be accepted with a value:
|
change device orientation | portrait, landscape |
Device orientation can be changed to portrait/landscape on android/iOS device:
|
Reusable Rules (Subroutines)
If you have a sequence that will be used often, you can save them as a Reusable Rule and refer to it with the name of your choosing.
Example: The first 6 steps of the example below will take you to your checkout page. If you frequently perform this process, create one rule to simulate all of those steps.
login
click "Men Clothing"
scroll down
click "Men's cargo shorts"
click "brown"
click "Size 34"
click "Add to cart"
check that page contains "Your order is nearly complete!"
Then, go to the Reusable Rules section in testRigor, assign the name "go to checkout page" to your rule, and add the first 6 steps above. From then on, you can simply use the syntax below to trigger all 6 steps:
go to checkout page
check page contains "Add to cart"
You can also create rules with dynamic parameters.
Rule name:
search "product" click on the link and then press "button"
Steps:
(We're going to create variables with the same name you define between quotes.)
enter stored value "product" into "search"
click link stored value "product"
click stored value "button"
example
Then you can call the rule like:
Search "Computer" click on the link and then press "Add to Cart"
The variables defined are scoped and can only be accessed inside the rule.
Referencing Locations and Elements
All references must be in double quotes, which can be escaped by backslash. There are several ways to refer to elements (for checks, clicks, entering data, etc.).:
Attributes
The following list contains the supported attributes that testRigor can interact with. Attributes are searched for without the need to refer to them in the script. The command check that "peter" color is "ffaabb"
will search through the following attributes:
Attributes for Desktop web browser and mobile web browser testing
Desktop web browser and mobile web browser testing support the following attributes:
-
Nested text - example
- Placeholder - example
- Value - example
- data-tid/data-testid/data-test-id/Id/resource-id/data-id - example
- Name - example
- aria-label - example
- CSS Class - example
- Label from ML classification - example
- Hint/Title/Tooltip - example
- Alt/Src - example
- For inputs/edits/dropdowns/selects/etc. will also search corresponding label - example
*Note: In addition to the above mentioned attributes, custom attributes can be added for desktop web browser testing in Settings
->Advanced
->Custom attributes to consider for finding elements new line separated
Attributes for native mobile application testing
Native mobile applications support the following attributes:
Android
- content-desc
- class (usually named something like
android.widget.TextView
) - resource-id
- text/label
- accessibility-id
- XCUIElementType
- name
- text/label
*Note: Hybrid applications tested on testRigor infrastructure use the attributes both from web browser applications and native mobile applications.
Attributes for remote desktop application testing (only available if Windows Application Driver is properly set up)
- AutomationId
- Name
Generic Indexes
Generic indexes are allowed for when multiple instances of the same element are on the page. For example:check that second "peter" color is "ffaabb"
check that 2nd "peter" color is "ffaabb"
example
Types
Types allow us to differentiate elements that have the same name on the screen. For example, if there is both a button and an input placeholder named "Search" on the page and you want o click the button, the way to specify it would be
click button "Search"
. Supported types are:- text - example
- label - example
- button - example
- link - example
- input (or "edit" or "field") - example
- dropdown (or "select") - example
- checkbox (or "switch") - example
- radiobutton - example
- file input (or "input file" or "edit file" or "input type file"): specifically an input of type file - example
Typed indexes
Typed indexes are a combination of general indexes and types. For example,check that second input "peter" color is "ffaabb"
will find the second input/edit named "peter".
example
Stored value for controls
Stored value for controls is the use of a variable to find and element or control. For example,
example
check that stored value "peter" color is "ffaabb"
will resolve the stored value for "peter" and use that resolved value to find the control.Image class
Image class uses image recognition technology to identify common icons. For example,
example
click on "cart"
. testRigor classifies images on the screen and if there is a button which looks like a shopping cart it will click on it. Complete list of all button image categories is here.Stored value for data
For example,
example
check that "peter" contains stored value "actionNote"
will find the second input/edit related to "peter".All the ways to refer an element described above apply to all types of actions like check, enter, click, etc. The options above can be combined together.
Multiple references support
Multiple references allows us to use
or
in instances where the targeted element has two possible names. In certain cases, certain elements might be expected to have different names but mean the same thing. For example:click "checkout" or "submit"
example
It can also be combined with "if exists" to avoid failing if the element doesn't exist:
click "checkout" or "submit" if exists
example
Selecting elements in reference to other elements
You can select elements based on it's position in relation to some other element. For example, you can refer to elements in certain sections like:
click on button "Delete" below "Actions"
Supported relative locations are:
You can also refer to the elements by 2 references:
click on button "Delete" below "Section Name" to the right of "label"
example
In addition, you can specify how the relative position should be used:
completely
- default overlap with the target area is at least 30% - examplewith at least "10" percent overlap
- similar to above but overlap can be specified - exampleroughly
- element is anywhere below the cut line for "below", to the right of for "to the right of" - example
For example,
enter "Peter" into roughly below "Section"
enter "Peter" into element with at least "1" percent overlap on the right of "Description"
Note, that the default behavior is "completely". However, if there are 2 anchors specified, the first anchor defaults to "roughly" and the second one "completely". I.e.
enter "Peter" into "Section" below and on the right of "Description"
is equivalent to:
enter "Peter" into "Section" roughly below and completely on the right of "Description"
Selecting elements in the context of other elements
You can select elements within the context of other elements. For example, you can pinpoint row at the table and ask to click a button in the context of that row:
click on "Delete" within the context of table "actions" at row containing "id1" and column "Actions"
example
You can use references to other elements (previous section) in the context itself
click on "Delete" within the context of "sectionTwo" below "Actions" and to the right of "rowName"
Selecting elements using reference to other elements and in the context of other elements
You can narrow where to look for an element using a combination of references to and in the context of other elements.
- Context is processed first, narrowing what to consider to what is inside that element
- References to other elements is processed next, only considering what was delimited by the context
click on "Button" below "Title" in the context of "sectionOne"
example
click on "Button" below "Title" and to the right of "leftHeader" in the context of "sectionTwo"
example
You can even specify up to 4 references to other elements using this combination
click on "Button" below "Subtitle" and to the right of "leftHeader" in the context of "sectionOne" below "Title" and above "Second Title"
example
Specifying the type of an element
You can force the system to only deal with a certain type of elements. The following command will not click on any text that is not a button or link:
click on strictly button "Delete"
example
Case sensitivity and exact matches
You can choose to deal with only "exactly" matches (i.e., case sensitive full strings match, or just say "case sensitive" to make matching case sensitive):
click on exactly "Delete"
example
It can be combined with the strict type selection like so:
click on strictly button exactly "Delete"
example
All supported modifiers to a string being searched:
case sensitive
exactly
(case sensitive full string must be exactly the same)full string
(not necessarily case sensitive)contains template
contains substring which matches a simple templatecontains regex
contains substring which matches a regular expressionmatches template
the full string must match a simple templatematches regex
the full string must match a regular expressionstrictly less
lexicographically strictly earlier (empty strings are always not less)less or equal
lexicographically earlier or equal (empty strings are always not less or equal)strictly more
lexicographically strictly later (empty strings are always not more)more or equal
lexicographically later or equal (empty strings are always not more or equal)
Fine-tuning access
In some cases, the target might be too wide, in this case you can force it to go deeper:
click deepest element "Delete"
example
It will issue click on the deepest element in the element tree that has the text.
Specifying position on screen
It is possible but highly discouraged (for stability reasons) to use offsets to click on specific part of an element:
click on "Delete" with offset "20,10"
offset is calculated from top left corner of the element, horizontal coordinate first. It is also possible to specify some generic positions like:
click in the middle of the screen
The supported positions are:
in the middle of the screen
- examplein the top quarter of the screen
- examplein the second top quarter of the screen
- examplein the bottom quarter of the screen
- examplein the second bottom quarter of the screen
- examplein the top third of the screen
- examplein the bottom third of the screen
- examplein the left quarter of the screen
- examplein the second left quarter of the screen
- examplein the right quarter of the screen
- examplein the second right quarter of the screen
- examplein the left third of the screen
- examplein the right third of the screen
- examplein the left side of the screen
- examplein the right side of the screen
- examplein the top of the screen
- examplein the bottom of the screen
- exampleon the left edge of the screen
- exampleon the right edge of the screen
- exampleon the top edge of the screen
- exampleon the bottom edge of the screen
- examplein the left top corner of the screen
- examplein the right top corner of the screen
- examplein the left bottom corner of the screen
- examplein the right bottom corner of the screen
- example
Saved values (variables) support
testRigor supports generating random values based on RegEx , saving them and using them later in the test. For example,
generate from regex "[a-z]{10,18}", then enter into "Notes" and save as "actionNotes" //example
validate that stored value "actionNotes" color is "ffaabb" //example
check that "peter" contains stored value "actionNote" //example
click on stored value "actionNotes" //example
check that page contains stored value from "actionNotes" //example
There are 2 special stored values:
"username"
and "password"
which come from Application-specific credentials settings for your test suite. With it you can do:enter stored value "username" into "username_field" //example
enter stored value "password" into "password_field" //example
These stored credentials are also used by
login
command. Also, you can use stored variables as parameters in most commands:enter from the string with parameters "${homePrefix}/my/path" into "urlPath"
example
And calculate expressions like this:
check that expression "${a} + ${b}" itself is equal as a number to "42"
example
testRigor supports ECMAScript 5.1 compatible expressions.
For example, you can calculate to check that a year from 30 days ago is present on the screen:
check that page contains expression "var aDate=new Date();aDate.setDate(aDate.getDate()-30);''+aDate.getFullYear()"
testRigor supports the following pre-defined saved values:
- username - example
- password - example
- homeDomain - example
- homeFile - example
- homePrefix - example
- todayYear - example
- todayYearShort - example
- todayMonthNumber - example
- todayMonthNumberTwoDigits - example
- todayMonth - example
- todayMonthShort - example
- todayDayOfMonth - example
- todayDayOfMonthTwoDigits - example
- todayDayOfWeek - example
- todayDayOfWeekShort - example
- nowHour - example
- nowHourTwoDigits - example
- nowMinute - example
- nowMinuteTwoDigits - example
- nowSecond - example
- nowSecondTwoDigits - example
- nowNanosecond - example
- nowDateIso - example
- nowTimeIso - example
- nowDateTimeIso - example
- nowMillisecondsFrom1970 - example
- nowDateTimeRFC1123UTC - example
- nowUnixTime - example
- testSuiteParentFolder - example
- testSuitePath - example
- testSuiteName - example
- testCaseName - example
- testCaseExecutionLink - example
- currentUrl - example
testRigor also supports grabbing values from elements on the screen and saving them into variables for later usage. For instance:
grab value from "some-element" and save it as "my-email"
grab value from input "some-edit" and save it as "my-email" //example
You can also grab a set of values from a table row or column (the value will be stored as a JSON Array)
grab values from table "some-table" at column "some-column" and save it as "some-column-values"
grab values from table "some-table" at first row and save it as "first-row-values"
You can set variables directly without entering it anywhere. For instance:
generate from regex "[a-z]{10,18}" and save as "actionNotes" //example
save value "Peter" as "name" //example
Executing actions in a loop
testRigor has limited support for executing commands until a certain condition is true. For example,
click "Next" until page contains stored value "previously generated id"// example
It can be used for going through pages in long lists or scrolling down until a certain text is visible or certain button is visible. By default the action will be executed up to 10 times, but you can extend maximum number of times by adding "up to":
click "Next" up to 12 times until page contains strictly button "Place order" // example
You can do the same thing with rules as well:
Go To The Next Page until page contains stored value "previously generated id"
Do Something up to 12 times until page contains strictly button "Place order"
Validations
Validations are supported for multiple things like:
- Finding something on the screen/page
- Validating properties or content of an element/finding element with given properties and content
- Visual validations (colors, matching images to of elements to later versions)
- Downloaded files validations
- Audio validations (Ubuntu only)
- Video validations (desktop web browser testing only)
- Email validations
- SMS validations
- Phone call validations
- Mathematical validations/calculations of formulas (even for data that just looks like financial data like -$30,000.23)
- Built-in API validations
- Chrome Extensions Testing
- Etc.
For UI validations for the whole page we support (positive and negative):
- contains/doesn't contain (text). For example,
check that page contains "Error"
- contains template/doesn't contain template (text). For example,
check that page contains template "###-###-####"
- contains regex/doesn't contain regex (text). For example,
check that page contains regex "(+\d )?\d{3}-\d{3}-\d{4}"
- return code is. For example,
check that page return code is "404"
- did not change compared to the previous step. For example,
check that page did not change compared to the previous step
- url starts with/url doesn't start with. For example,
check that url starts with "https"
- url contains/url doesn't contain. For example,
check that url contains "testrigor.com"
- url is/url is not. For example,
check that url is "https://testrigor.com/docs/language/"
- url ends with/url doesn't ends with. For example,
check that url is "https://testrigor.com/docs/language/"
- url matches regex/url doesn't match regex. For example,
check that url matches regex "https://testrigor\.com/docs/\w+/"
- title is. For example,
check that page title is "testRigor - Documentation"
- title contains. For example,
check that page title contains "testRigor"
For UI validations for elements we support (positive and negative):
- contains/doesn't contain (text) - example/example
- is blank/is not blank - example/example
- matches regex/doesn't match regex - example/example
- matches simple template/doesn't match simple template -
check that "section1" matches simple template "$########"
- contains simple template/doesn't contain simple template -
check that "section1" contains simple template "$########"
- contians regex/doesn't contain regex -
check that "section1" contains regex "[A-Z][a-z]+"
- lexicographically before -
check that "section1" lexicographically before "zzzz"
(empty string is not lexicographically before anything) - lexicographically before or the same as -
check that "section1" lexicographically before or the same as "zzzz"
(empty string is not lexicographically before or the same as anything) - lexicographically after -
check that "section1" lexicographically after "AAAA"
(empty string is not lexicographically after anything) - lexicographically after or the same as -
check that "section1" lexicographically after or the same as "AAAA"
(empty string is not lexicographically after or the same as anything) - has value/doesn't have value (for inputs/text areas) - example/example
- is checked/is not checked - example/example
- is disabled/is enabled - example/example
- is visible/is invisible - example/example
- color is - example
- is clickable/is not clickable - example/example
- cursor is - example
- has CSS class - example
- background color is - example
- has property - example
- has options selected (for selects) - example
check that "element" color is "ffaabb"
check that input "input" has value "value"
check that checkbox "Keep me signed in" is checked
check that checkbox "Keep me signed in" is not checked
Validate that an input has no value:
check that input "input" has value ""
Validate that element has some property:
check that property "background-color" of "my-div" is equal to "black"
check that property "width" of "my-div" is equal as a number to "310px"
check that property "height" of "my-div" is greater or equal than "170"
For stored values, including: API return value validations, values grabbed from the screen, values extracted from text. We support (positive and negative):
- matches regex/doesn't match regex - example/example
- contains/doesn't contain - example/example
- is equal to/is not equal to - example/example
- is null/is not null - example/example
- is blank/is not blank - example/example
You should use the word
itself
to perform validation on a stored value:call api "https://testrigor.com" and save it as "variableName"
check that stored value "variableName" itself contains "James"
example
The key to the testing API results is to save it to stored value and then perform validation on stored value itself (with keyword itself
) as shown above.
Both element and stored value validations support:
- is equal as a number to/is not equal as a number to - example/example
- is greater than - example
- is greater or equal than - example
- is less than - example
- is less or equal than - example
You can also enjoy validations that apply to the whole screen/page:
- page contains/page doesn't contain - example/example
- page has regex/page doesn't have regex - example/example
- page's return code is
- title is - example
- title contains - example
- url is/url is not - example/example
- url contains/url doesn't contain - example/example
- url starts with/url doesn't start with - example/example
- url ends with/url doesn't end with - example/example
- url matches regex/url doesn't match regex - example/example
For example, you can combine stored values for validation:
check that url matches regex from the string with parameters "${homePrefix}/product/${type}/[0-9A-Za-z\-]+"
Visual Testing
You can verify visual changes on the screen using the following sample steps:
compare screen
compare screenshot to previous version
compare screen to previous version with allowance of "1%" treating error as "minor"
compare screen to previous version treating error as "minor"
It is possible to save a sample screenshot in test data and use it as the point of comparison instead of the previously saved screen image:
compare screen to stored value "Saved Screenshot"
Regarding the severity parameter, the allowed values are "minor", "major", "critical", or "blocker". The default value is "critical", which marks the test case as failed.
Similarly, there is an option to use the "allowance" clause, a minimum acceptable difference in percentages. Any difference below this value will be ignored. You may need to experiment with this parameter. In any case, you can find actual discrepancy in "Extra info" for a particular step.
On mobile devices we ignore the status bar on top. This is so because there is a time stamp that may be different for each screenshot.
For this task you can also use Applitools. To enable Applitools integration you need to configure and enable it first. There is a documentation guide with the steps to do it. To use the algorithm in tests you need to add "using Applitools"
clause. E.g.:
compare screen using Applitools
compare screenshot using Applitools
compare screens "Screen Name" using Applitools
compare screen using Applitools treating error as "minor"
compare screens "Screen Name" using Applitools treating error as "major"
The screen name and the severity of the error when the screens mismatched are optional.
You can use the screen name to identify the screenshot in the Applitools application. If you didn't put the name it will be called as "Image n", where n is the index of the screen in the test case. For instance, you can use this command 3 times in one test case, this way names would be "Image 1", Image 2" and "Image 3".
If Applitools identify a mismatch in the screen comparison, the test case will be marked according to severity and a link to the Applitools report will be provided.
Applitools always compare the current screenshot with itself baseline. You can set the baseline of each screenshot in the Applitools application. If you don't do it, the first version of each screen would be considered it's baseline.
Clicking on specific images with visual testing
testRigor also offers visual support for clicking on images by indexing for instances where images or icons do not have a simple way to be referred to. To accomplish this, crop the desired image from a screenshot or file in its full resolution and save it in the Test Data Section. To then refer to it in your case, use the syntax below:
click on the 6th element by image from stored value "logo" with less than "10" % discrepancy
If there is only one image that you want testRigor to recognize, use the following:
click by image from stored value "logo" with less than "10" % discrepancy
You may need to toggle discrepancy percentages in order to allow testRigor to recognize the image.
Working with Tables
You can refer to table cells by the intersection of the row and column by providing value of the first cell in the row and value of the header/top cell in the column. For example, command:
click on table "actions" at row "103" and column "Action"
example
For the table
# | Id | Name | Actions | Additional Data |
---|---|---|---|---|
Filter by | ||||
101 | york1 | Yorktown | Arrive Cancel | |
102 | spk2 | Spock | Listen to Ignore | |
103 | nyo3 | Nyota | Open channel Promote |
You can also specify row by saying that row should contain a certain value. This way we will check all values of every row to find the one which matches. For example, for the same table above, and command:
click on table "actions" at row containing "spk2" and column "Actions"
example
will result in a click on the first action "Listen to". To click on second action "Ignore" you can leverage our context feature:
click on "Ignore" within the context of table "actions" at row containing "spk2" and column "Actions"
example
You can also work with multiline headers by referring to them wither "header" word:
enter "york1" into first table at the second header row and column "Id"
example
testRigor supports tables for all kinds of operations including but not limited to: validations (checks), clicks, hover, entering data (enter … into), drag and drop, etc. Examples:
check that table "actions" at row "102" and column "Name" contains "Spock" //example
check that table "actions" at row "101" and column "Additional Data" has value "Looks like a trap" //example
enter "This is a trap!" into table "actions" at row "101" and column "Additional Data" //example
click "Open channel" within the context of second table at row "103" and column "Action" //example
check that the second table at row containing "Nyota" and column "Action" contains link "Open channel"
click the first button within the context of second table at row containing "Nyota" and column "Action"
Row or column value aggregation
You can check certain aggregations and comparations with the data contained in an entire row or columnOrder of values
To check the order of the values contained in a row or columncheck that table "actions" at column "Name" has values sorted in ascending order
check that table "actions" at column "Name" has values sorted in descending order
Number of values (count)
To check the value count in a row or columncheck that table "actions" at column "Name" the value count is greather than "2"
check that table "actions" at column "Name" the value count is less than "2"
check that table "actions" at column "Name" the value count is equals to "2"
Sum of values
To check the sum of values in a row or columncheck that table "actions" at column "#" the sum of values is greater than "100"
check that table "actions" at column "#" the sum of values is less than "500"
check that table "actions" at column "#" the sum of values is equals to "306"
Average of values
To check the average of values in a row or columncheck that table "actions" at column "#" the average of values is greater than "100"
check that table "actions" at column "#" the average of values is less than "300"
check that table "actions" at column "#" the average of values is equals to "155"
Conditional execution
testRigor supports conditional execution of command or a rule. For a command it will look like this:
click "element" if page contains "bla" //example
enter "macbook" into "search" if "Search in" has value "bla" //example
In certain cases, certain elements might appear randomly on the screen. For such cases, there is an
if exists
clause. For example,click "element" if exists //example
enter "bla" into "element" if exists //example
The above commands will not fail if element is not found and will be skipped silently.
Conditional Rules
You can also make execution of a rule conditionally like this:My Rule if page contains "bla"
Purchase Product if "Search in" has value "Products"
There is a way to fail at will as in the following example:
fail //example
fail with "error" //example
API Testing
testRigor supports calling to API's, getting value and saving result as a stored value:
call api <TYPE> "<API_URL>" with headers "a:a" and "b:b" and body "body" and get "JsonPath" and save it as "variableName"
For example,
call api post "http://dummy.restapiexample.com/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body "{\"name\":\"James\",\"salary\":\"123\",\"age\":\"32\"}" and get "$.data.name" and save it as "createdName" and then check that http code is 200
example
In the example above, testRigor would validate that return http code is 200. Parameters
"JsonPath"
and "variableName"
are both optional. However, if "JsonPath"
is present, "variableName"
will be mandatory. Here is one more example,call api "api" and save it as "variableName"
JsonPath is the way to refer to parts of JSON and described with examples here. If "JsonPath" is not defined in the action, the complete result of the request will be stored in "variableName".
testRigor supports all HTTP actions for call <TYPE>:
- get - example
- post - example
- put - example
- patch - example
- head - example
- delete - example
- options - example
- trace - example
with headers "a:a" and "b:b"
You can pass JSON into the body of POST message, with double quotes characters escaped like so: "\"". You could use tools like this to escape it for you. For example, and body "{\"name\":\"James\",\"salary\":\"123\",\"age\":\"32\"}"
You can use parameters and multi-line strings for constructing a call:
call api post from the string with parameters "${homePrefix}/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body text starting from next line and ending with [END]
{
"param": "value",
"param2": "value2"
}
[END] and get "$.data.name" and save it as "createdName"
And you can pass parameters:
call api post from the string with parameters "${homePrefix}/api/v1/create" with headers "Content-Type:application/json" and "Accept:application/json" and body from the string with parameters text starting from next line and ending with [END]
{
"param": "value",
"param2": "${dynamicValue}"
}
[END] and get "$.data.name" and save it as "createdName"
After the API call is executed, if the result was stored into saved value it could be later tested like so:
check that stored value "createdName" itself contains "James"
Mocking API calls
testRigor supports mocking reponse data (headers, body and/or http status code) for api calls made inside your application.
mock api call <TYPE> "<API_URL>" returning body "<MOCK_BODY>"
For example,
mock api call get "https://dummy.restapiexample.com/api/v1/employees" with headers "a:a" returning body "This is a mock response" with http status code 200
In the example above any get calls to the endpoint "https://dummy.restapiexample.com/api/v1/employees" with the headers "a:a" will reponse the testRigor mock with status 200.
Some useful cases are:
- You might want to use mocking apis if you are using third party api calls, those calls can be charded and expensive so you can mock the responses insted of call the real service.
- You can test your application individually if servers are instable or down.
- You can test specific scenarios, for example if you want to test a scenario where de server returns error.
Uploading files
File upload is supported out of the box. Just use it like the following:
enter stored value "keyName" into input file "fileField" //example
enter "<FILE_URL>" into file input "fileField" //example
Where "fileField" is a file input element, i.e.: input type="file" ...
You can upload the file into testRigor store in "Test Data" section, then use it by name as stored value. Alternatively you can use your own URL. If you choose to upload from your own URL, the link should be downloable. In any case the file is uploaded from within testRigor environment.
You can upload a file into an Android mobile device using the following action:
upload file from saved value "sampleFile"
upload file "https://some-page.com/path-to-file" to mobile device
The file will be uploaded into
Downloads
folder on the device:Multiple files
Multiple files upload can be done like so:
enter from the string with parameters text starting from next line and ending with [END]
${file_var_one}
<FILE_URL>
[END] into file input "myFileInput"
example
Working with folders
If the input element has awebkitdirectory
attribute, e.g. - <input type="file" webkitdirectory ...
this element is expecting a folder. If you try uploading a file, we will put it into a temporary folder and enter that value. However, if you want to upload an actual folder, you can zip the folder and use it as an input. In such case, we will unzip the file, preserving directory structure and enter the root path. This is similar to the drag folder action. Note: This does NOT work in headless mode. It also does NOT work with Internet Explorer
Template files
You can also specify files containing comma separated values (CSV) and parameterize those with variables (any stored value) in the form ${nowDateIso}.
The file should be in CSV or TXT format and contain the values and variables in the form:
some value, ${variable1}, another value
${variable2}, fixed value, ${variable3}
example
To allow templating you need to use the word "template" like the following
enter template stored value "keyName" into input file "fileField"
enter template "<FILE_URL>" into file input "fileField"
example
Phone calls
testRigor supports making a call through Twilio . There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for making calls.
After the integration is setup you can add custom steps like:
call "+15344297154" and validate it was picked up //example
make call to "+15344297154" and check it was answered //example
call to +15344297154 //example
call "+15344297154" and check it was completed //example
call "+15344297154" from "+15551234567" and verify it is ringing //example
check that phone call from "+15344297154" is ringing //example
check that phone call from "+15344297154" was answered //example
Number to call from is optional and may need to be allocated prior to using. Numbers to call from and to can be from stored values or parametrised strings:
call from stored value "allocatedNumber" to stored value "answerPhoneNumber"
example
SMS messages (Phone Text)
You can send messages thanks to Twilio integration. There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for sending messages.
Then you can add custom steps like:
sms "+15344297154" with body "this is content" and validate it was sent //example
send sms to "+15344297154" with content "this is content"
send message to "+15344297154" with text "this is content" and check it was delivered //example
send message from +15551234567 to "+15344297154" with text "this is content" and check it was failed
message from +15551234567 to "+15344297154" with body "this is content" and check it was not delivered
Number from which to send message is optional and may need to be allocated prior to using.
You can also check SMS messages and validate and store their contents:
check that sms from "+12345678901" to "+12345678902" contains "Code" and matches regex "Code\:\d\d\d\d" and save it as "sms"
You can then additionally apply
extract value
command to get just the code:check that sms to "+12345678902" matches regex "Code\:\d\d\d\d" and save it as "sms"
extract value by regex "(?<=Code\:)[0-9]{4}" from "sms" and save it as "confirmationCode"
Number from, number to, and body can be from stored values or parametrised strings. Additionally, message body may be a multiline string:
send sms from stored value "allocatedNumber" to stored value "answerPhoneNumber" with content from string with parameters starting from next line and ending with <END>
${answerCode} b
<END>
example
You can request a temporary phone number from Twilio . There is a section in the application configuration under Integrations for setting Twilio parameters; it is required for making calls.
After the integration is setup you can add custom steps like:
allocate a temporary number and save it as "newNumber"
This custom step will request a new phone number (Twilio charges will apply) and save it in a variable "newNumber", which you can use to check for incoming calls or messages. When the test run finished, this number will be released automatically.
Here is an example of testing a 2FA login with SMS:
click "Sign in"
enter "jacob" into "Username"
enter "jacobs-secure-password" into "Password"
click "Verify me"
check that sms to "+12345678902" is delivered and matches regex "Code\:\d\d\d\d" and save it as "sms"
extract value by regex "(?<=Code\:)[0-9]{4}" from "sms" and save it as "confirmationCode"
enter saved value "confirmationCode" into "code"
click "Continue to Login"
check that page contains text "Welcome, Jacob!"
Login support
testRigor supports login with a single command like so:
login
example
This action identifies and performs the necessary steps required to login to the application automatically. After running successfully the first time, it creates a rule named Autogenerated Login for the application containing the identified steps, that you can override by creating a reusable rule with the same name.
For example, for an application with a simple email and password login, the steps executed and included in the rule would be:
enter stored value "username" into "email"
enter stored value "password" into "password"
click "Log In"
Note that login command relies on having login credentials configured for your application on testRigor.
Email testing
testRigor supports testing with both sending emails as well as receiving and validating emails:
send email to "[email protected]" with subject "Test message" and body "Hi, this is a test, this is just a test message."
check that email from "[email protected]" is delivered
example
Sending emails also supports sending attachments by referencing a URL of a file you'd like to attach:
send email from "[email protected]" to "[email protected]" with subject "Test message", and body "Hi, this is a test, this is just a test message.", and attachment from saved value "Sample File"
example
send email from "[email protected]" to "[email protected]" with subject "Test message", and body "Hi, this is a test, this is just a test message.", and attachment "http://online.com/file/name.pdf"
example
Notes:
- "to" and "from" can be used at the same time. If not specified, "from" address will default to "[email protected]".
- A stored file defined in "Test Data" can be used as an optional attachment. An attachment can also be a link to a file available online without username/password.
- Both "from" and "to" addresses can be filled from stored values (e.g., "to saved value "newEmail"" and/or "from saved value "newEmail"")
Checking email will automatically open and render it as an HTML page in a desktop browser window:
check that email to "<random-user>@testrigor-mail.com" from "[email protected]" is delivered //example
check that two emails to "[email protected]" and "Confirm" in subject were delivered //example
check that one or more emails to "[email protected]" and "Confirm" in subject were delivered //example
check that email to saved value "newEmail" was received //example
check that email to saved value "newEmail" from "[email protected]com" and "Confirm" in subject was received //example
check that email to saved value "newEmail" and "<regular expression>" in subject was received
If you need to render the message in the browser of the same mobile device (Android only) where you test your app of website (if tested in a device), e.g. - when validating a sign up test case, or just want to see how it looks on a small screen, you can mention this in the action like so:
check that email to "<random-user>@testrigor-mail.com" and "Confirm" in subject was received and show in mobile
example
In some cases you may need to check that email was not sent, here is how to do this:
check that email to "<random-user>@testrigor-mail.com" from "[email protected]" was not delivered
example
Notes:
- If the email count is not specified, we assume that only one message is expected. Multiple emails are treated as an error; so in order to avoid receiving the error message, specify the number of emails that should be expected or simply inform the system that more than one can be sent ad done below:
examplecheck that one or more emails to saved value "<random-user>@testrigor-mail.com" were received
- The user should send a message to the following address:
<random-user>@testrigor-mail.com
, where<random-user>
cab be any valid email handle. In order to avoid a conflict it is recommended that the user handle will contain customer name and some random part, e.g. [email protected] This is important when you are running email tests in multiple browsers and/or on multiple machines at the same time. In this case it is your responsibility to ensure that you are not getting messages, meant for another test case run. - The user is expected to add some wait time before checking email. This check action will validate delivery, but not wait for the message to arrive. For example,
wait 2 minutes
- Once the action has been executed successfully, the message is deleted from the mailbox and rendered in a new tab. Multiple emails will be marked as error, but each one will be rendered in a separate tab. You can switch between tabs using
switch to tab "1"
,switch to tab "2"
, etc. Note that old content will stay in the tab before the first rendered email message. In most cases it is tab "1", unless you opened more tabs. - Each email tab is rendered as an html page, so you can use any actions that you would use on an html page. There are a few useful element ids that will help getting specific parts of an email:
- "message-froms" a list of "From" addresses (usually one address)
- "message-subject" subject line
- "message-date" message sent date and time in Pacific Time zone.
- "message-tos" a list of "To" addresses
- "message-ccs" a list of "CC" addresses
- "message-bccs" a list of "BCC" addresses
- "message-reply-to" "Reply To" address
- "message-text" the message text
- "message-attachments" a list of attachments (file names only)
- If you want to check again, you need to re-send the mail. Your custom actions cannot contain multiple "check" actions for the same message.
-
Multiple recipients can be separated with "and":
examplesend email to saved value "newEmail" and saved value "newEmail2" with subject "Test" and body "Hi"
- When checking emails we only take into account those sent after a specific test case has started. This is to avoid a conflict when an email was left unchecked from a previous, unfinished run.
click "Sign up" generate unique email, then enter into "Email" and save as "generatedEmail" generate unique name, then enter into "Name" and save as "generatedName" enter "PasswordSuperSecure" into "Password" click "Submit" check that email to stored value "generatedEmail" was delivered click "Confirm email" check that page contains "Email was confirmed" check that page contains expression "Hello, ${generatedName}"
Browser cookies, localStorage, sessionStorage, userAgent
The user can set/get/clear browser cookies like this:
set cookie "cookie value" as "cookie-name" //example
set cookie from saved value "variableName" as "cookie-name" //example
get cookie "cookie-name" and save it as "variableName" //example
clear cookies //example
set/get/clear browser localStorage and sessionStorage items like so:
set item "item-data" in sessionstorage as "item-name" //example
get item "item-name" from session storage and save it as "varName" //example
clear sessionstorage //example
set item "item-data" in localstorage as "item-name" //example
get item "item-name" from local storage and save it as "varName" //example
clear localstorage //example
and set/unset a custom userAgent value like this:
set user agent to "My User Agent" //example
unset user agent //example
Audio testing
You can test audio by recording it and then comparing it with another recording in the same test or an external file. (Audio testing is currently available for Linux/Ubuntu devices only.)
Recording
You have the option to record all the audio that is being reproduced in a tab. The command below in an example:
record audio through 10 seconds after clicking "audio-trigger" and save as "my-recording"
You can also record the audio for a specific audio tag as in the example below:
record audio from "my-audio-tag" through 20 seconds after clicking "audio-trigger" and save as "my-specific-recording"
Comparing
After having a recording that is saved as a variable, you can compare it against another recording. For example,
check that audio from "my-recording" is "70%" similar to "my-specific-recording"
You can also compare it against an external file:
check that audio from "my-recording" is "85%" similar to "https://some-page.com/path-to-file"
The supported file extension for external files is .wav
When comparing we can test positive or negative by using similar or different, for example,
check that audio from "my-recording" is "99%" similar to "my-specific-recording"
or
check that audio from "my-recording" is "1%" different to "my-specific-recording"
Playing
If you need to reproduce audio as it was someone speaking through a microphone you could load the .wav that is going to be reproduced from a remote file. For tests running in linux, virtual devices are used (microphone/speakers) otherwise we attach to existing < audio > tagsYou can reference a remote file to download
play audio "https://some-page.com/path-to-file"
Or a reference from test data
play audio "test-data-ref"
If more control about when to play the audio and when to stop it is needed, you can use
play audio "sound-to-play-ref" after clicking "start-recording" then click "stop-recording"
Validating
If you need to check that audio is produced (e.g., that a video is playing correctly with sound), you can use the following commands:
check that audio is playing
Negative test is also available:
check that audio is not playing
In both cases, a 10-second audio sample will be recorded, which will be available in the artifacts for download for manual validation if needed.
SQL Testing
You can use SQL in the steps for getting or inserting rows.
Select
The following example will get the first row and translate each pair of column-name/value into a stored value with the column name as the key:
run sql query "select top 1 UserID, LastName, FirstName from Users;"
example
Then you can use the stored values like:
enter stored value "FirstName" into "First name"
check that stored value "LastName" itself contains "Doe"
example
Insert
Executes the defined insert statement, for example:
run sql query "insert into Users(UserID, FirstName, LastName) values (3, 'Jon', 'Doe');"
example
Multiple Connections
It's possible to configure more than one connection. In this case, you can indicate the connection to be used by name:
run sql query "select top 1 UserID, LastName, FirstName from Users;" using connection "connectionName"
Note: If the connection name is not passed, the application will use the first configured.
Overriding the settings
You can update the connection settings at runtime using the following variables: "connectionName:usernameJDBC", "connectionName:passwordJDBC" and "connectionName:connStringJDBC":
Save value "user" as "connectionName:usernameJDBC".
Save value "pass12345" as "connectionName:passwordJDBC
Save value "jdbc:mysql:host:port" as "connectionName:connStringJDBC"
run sql query "select top 1 UserID, LastName, FirstName from Users;" using connection "connectionName"
Chrome Extensions Testing
You can use your extension to test it or test the integration with your/other pages.
A CRX file of the extension is needed for this feature. Upload it in Settings -> Integrations -> Chrome Extensions, and assign a name to it as it will be needed for its use in the commands to open the extension.
To open the extension in a new tab, use the following command:
open extension "myextension"
To open the extension in a pop up or to simply enable the extension, you can use this command:
activate extension "myextension"
Comments support
//
like so: