My Certificate Find a Certificate About Certification QA Terminology Basics of Software Testing Basics of Test Automation QA Processes Overview Basics of testRigor BDD & SDD Web Testing Overview Mobile Testing Overview Native Desktop App Testing Overview API Testing Overview Click / Drag / Hover Entering Data Variables Validations Scrolls and Swipes Dealing with Dates and Times Dealing with Emails Working with Tables Conditional Sequences Cycles & Loops Reusable Rules SMS and Phone Calls Managing Multiple Environments Settings Integrations AI and Prompt Engineering Final Exam

Generative AI-Based Testing Certification

Click / Drag / Hover

Click / Drag / Hover

In this section, we will explore different types of clicks that you can use in testRigor as well has drag and hover actions.

Click

The most basic command in testRigor is the click action. Users can double click, long click, right click, and middle click.

Example:
enter "doll" into "search"
click "search"

testRigor is smart enough to understand that the first “search” refers to an input whereas the second one refers to a button.

Note: Parameters of all commands by default are not case sensitive and will find the element even if they match partially.

For example, click "search" will click on the same button as click "Search".
To click on button “a” in a web application, you can type:
click "a"
In a mobile application, you can use tap instead of click:
tap "a"

However, click works if you’d like to reuse the same steps across web and mobile applications.

Read more about click commands here.

Click with relative location

In general, testRigor does an excellent job identifying elements by its name or the text it contains as it appears on the screen; however, in some cases (e.g., when there are multiple instances of the same element on the screen), you will need to specify a relative location to provide a better context to identify it.
When working with relative locations, testRigor refers to the desired element as the target, and the element used to refer to the target via relative locations is called the anchor. In the following example, Read The Story is the target, to the left of is the relative location, and Release Manager - Edcast is the anchor.
click "Read The Story" to the left of "Release Manager – Edcast"
Nowadays, websites organize elements in compartments or containers commonly known as divs. Divs can contain several elements and/or smaller divs. In the next example, the navigation menu is a div, and the menu items are the elements inside that div. We can use the relative location in the context of to target an element inside it. Again, this is especially helpful when there is more than one instance of the same element on the page. Limiting the search to a specific div is a good alternative as a relative location since many elements are more likely to stay inside a div when relocated. This can help reduce maintenance in the long term:
click "Request a demo" in the context of "menu"

Directional relative locations

By default, a directional relative location (on the right, on the left, above, below) means “directly” (directly below, directly above, etc).
check that page contains "logo" below "See case studies"

If we add the keyword roughly, the meaning expands to “anywhere on the page” (“roughly below” means “anywhere on the page below”)
check that page contains "splunk" roughly below "See case studies"

Sometimes, the target element is not entirely inside of the search zone (the yellow lines extending from anchor), but rather overlaps with it. When 30% of the target element is inside the search area, testRigor still considers it to be in the direction specified. However, if less than 30% of the target element is located inside the search area, we can still target it by lowering the overlap percentage using the keywords with "20" percent overlap after the target element and before the relative location. The percentage parameter can be any number lower than 30.
click "splunk" with "20" percent overlap below "See case studies"

Note: The phrase roughly below is not appropriate because testRigor would likely click on the sentence above the line of logos because it also contains the word “Splunk”. Because less than 20 percent of the sentence is inside the search area, testRigor will not consider the sentence when we use the phrase with "20" percent overlap below.

Multiple relative locations

testRigor also allows the use of multiple relative locations in a single command:
click "element1" below "element2" and on the right of "element3"
It’s important to understand that multiple relative locations have a specific behavior. When combined with a second relative location, the first one automatically becomes “roughly” (i.e., “directly” becomes “roughly”). For example, testRigor will automatically translate click the element below "Additional Data" and on the right of "Nyota" to mean click the element roughly below "Additional Data" and directly on the right of "Nyota". This command will therefore click on the “Open channel” button. If the desired element is the empty field on the far right of “Nyota”, we would need to use the keyword completely to make the first relative location keep it’s default meaning “directly”:

click the element completely below "Additional Data" and on the right of "Nyota"

Read more about relative locations here.

Clicking with a mouse vs. using JavaScript

Many elements of a website are not alone but are encased inside areas or containers as mentioned above. These containers (known as divs in HTML) organize elements in layers. By default, clicks are processed via a Javascript event because they can target an individual layer. Mouse clicks are processed by actually pressing the mouse button and target the entire object like humans do. Since Javascript does not have to move the mouse to click on the element, it saves a few milliseconds, which can add up when cases have a lot of steps.

Despite JavaScript clicks being faster, mouse clicks are necessary in some cases. One such situation is when clicking on any non UI element. This usually comes in the form of browser popups asking for location, camera, or microphone permissions.

Some applications also prefer browser popups to UI popups for confirmation of actions performed in the UI.

Another case is when working with canvas-based platforms like Flutter or with text that is inside an image or canvas. For all scenarios requiring mouse clicks, these elements are found using OCR only. JavaScript clicks need the text to be in the DOM to locate it, but when the text is only in the image, OCR is the only option.

You can specify your preference as to which type of click should be the default in the settings section. Settings -> Advanced -> in the Desktop Web Fine Tuning section below How to process clicks.

The default setting for click types can be changed, and it is possible to specify in test case steps where a click type opposite of the default is necessary.

Example:
click "Allow" using OCR only using the mouse
click "div" using javascript

Note: In headless (browser-less) mode, there are no visible UI elements. This means that clicking with JavaScript is the only option. If your test cases require using the mouse, do not enable headless mode.

Click by index

As a rule of thumb, relative locations are preferred over indexes. However, in some cases, referring to the item by its iteration on the screen (1st, 2nd, 3rd) might be necessary.

Sometimes there will be multiple elements with the same title on the page, for example multiple “Add to cart” buttons. You can reference them via a number, like so:
click on the 3rd "Add to cart"
Remember how there might be multiple focus areas on the page? You can easily combine indexes with relative locations:
click on the second "Add to Cart" in the section "Section 1"

Middle click and double click

The middle click feature was added to allow users to simulate clicking with the mouse wheel. In most cases, this will automatically open a link in a new tab; however, it is the browser and the programming of the UI that ultimately determines what this action will do.
Example:
middle click "here" on the right of "See more information"
Double click allows you to select or highlight a word that you need to perform an action on (such as copy, delete, drag, etc):
double click "username"

Right click

Like the middle click, different applications may use the right click for different purposes. For example, some applications prompt the default browser pop-up that will provide extra options, but others may prompt a custom menu of options like when you right click a test suite name on testRigor’s homepage. Furthermore, the type of item (link, button, text, video, blank area) will produce a different list of options.
Example:
right click "filename"

Click on specific coordinates

It is also possible to click on a specific coordinate relative to an entire page element or to a single element on a page. This is rarely recommended due to stability issues that may arise; however, clicking by coordinate (or offset) relative to an element is always more stable than doing so relative to the entire page. This is especially true in situations where coordinates are the only realistic option for interacting with a UI, such as when working with elements on a canvas (an area on a page or an entire page where the elements added usually do not appear in HTML and have no independent locators or properties by which they can be identified. Text elements might be identified by OCR, but the coordinate is essentially the only option for elements without text).
Example:
click "canvas" with offset "20,10"

The offset is calculated from the top left corner of the element/page to the bottom right. The first number is the horizontal coordinate. The higher the number, the further to the right the coordinate will be. The second number is the vertical coordinate. The higher the number, the further down the coordinate will be.

Offsets can also be used with other commands, such as drag and zoom.

Using AI to click on elements 

AI can be used to interact with UI elements by interpreting prompts you provide.

Example:

click "cart icon" using AI

In this case, "cart icon" is the prompt given to AI. The AI analyzes both the page source and the screenshot to locate and click the cart icon.

If multiple similar elements exist (e.g., several cart icons), you can add more context to your prompt to specify exactly which one to click.

Example:

click "cart icon on the right of red colored football" using AI

AI can also perform more complex actions, such as holding a key while clicking an element.

Example:

hold key ctrl and click "button" using AI

In this case, AI follows the same logic; using both the source and the screenshot to identify and interact with the correct button.

Drag

testRigor supports drag and drop for elements and files. Use “drag onto” command
drag "text1" onto "text2"
drag file "<URL>" onto "element"
drag file from saved value "File to upload" onto "element"
drag "link to file" into "droppable file section"
If you need to drag a folder with files, you need to zip it and upload in the Test Data section and use it like so:
drag folder from saved value "Zipped Folder" onto "element"
Note: In headless mode, the above might not work because there is no visible UI to drag elements and because dragging is a mouse-specific action in testRigor, which complicates interacting as there is no browser window.
As mentioned in the section immediately above, the drag command can be used with coordinates. Even better, users can drag to multiple points to create shapes.
For example, here’s how offsets can be used to draw on a canvas by dragging mouse to multiple points without releasing the click:
drag "canvas1" with offset "0,0" to "canvas1" with offset "50,0" via "canvas1" with offset "0,50" through "canvas1" with offset "50,50"

Using AI to drag elements on the UI 

When you are unsure of an element’s name or there is no identifiable text, only a visual area, you can use AI to locate that element or area on the page

Example:

drag "w3schools image" to "rectangle above w3schools image" using AI

In this example, “w3schools image” and “rectangle above w3schools image” act as prompts for AI.

AI is not responsible for performing the drag action itself, it is specifically used to find and identify the elements’ locations on the UI based on the descriptions provided. Once identified, testRigor executes the drag action.

Hover

You can use a hover command in testRigor the same way as when using a regular mouse. Use it to trigger specific UI actions where needed.

Example:
hover over "Account & Lists"
click "My account"

Using AI to hover over elements 

AI can also be used to hover over UI elements based on the prompt you provide.

Example:

hover over "cart icon" using AI

Here, “cart icon” is the prompt given to AI. The AI uses both the page source and the screenshot to locate the element’s position, and then testRigor performs the hover action on that element.

If there are multiple similar elements, you can add more detail to your prompt to target the specific one you want to hover over.

Example:

hover over "cart icon on the right of iPhone 128gb" using AI

Screenshot behavior with click, drag, and hover actions

It is important to know that each of these actions will always prompt a new screenshot as the viewport (area of the UI displayed in the screenshot) and/or elements inside the viewport are expected to change. Certified testers often use a generic hover to prompt screenshots where they would not be taken otherwise:

hover over the middle of the screen

Test your knowledge

click "splunk" with "20" percent overlap below "See case studies"
click "splunk" below "See case studies"
click "splunk" below the overlap of "See case studies"
click "splunk" with "20" percent overlap roughly below "See case studies"
(a) tap “a”
(b) click “a”
(c) touch “a”
(d) press “a”
(e) options a, b, and c
(f) options a, b, and d
(g) options a, c, and d
drag file from stored value "file" from "directory" into "element"
drag "localDirectoryLocation" into "droppable file section"
drag file "<URL>" onto "element"
push file "<URL>" onto "element"
click 2nd "Filename"
click 2nd "Filename" in context of double click
click 2nd "Filename" double times
double click 2nd "Filename"
© 2023 testRigor. All rights reserved.