FAQ

How is testRigor's test automation different from Selenium?

1. Tests are automatically generated based on our AI's mirroring of how your end users are using your application in your production environment. In addition, tests are produced to map your most important functionality out of the box. This is achieved by using our JavaScript library in your production environment to capture metadata around your users' habits within your app. We use this metadata to automatically generate & maintain behavior-driven tests for your most frequently used functionality & flows. We also add the possibility for tests of most important functionality based on common and industry-specific rules.
2. Our tests are in plain English, so it is possible to understand what is being tested and to search for all tests affecting certain buttons, inputs, etc. Using the same language support, you can also customize rules to fit your own internal verbiage.
3. Because testRigor tests are not dependent on XPath, they are ultra stable. Our customers are running thousands of tests multiple times a day without a single failure.
4. Our tests are adaptable. We not only extract and show unique issues, but we also identify and group together all the cases affected by the same issue and allow our customers to fix them in place all at once with quick and easy editing tools.

How are you different from Cucumber/Gherkin?

You can think of testRigor as Cucumber already implemented for your project. We have eliminated the need to write and maintain any underlying Selenium code, as our system uses an NLP-based parser to parse plain English and executes your specifications in place.

What are your end-to-end tests actually testing for?

Acceptance-level functional UI-level regression tests. We test that functionality works via UI. For example, for Amazon we would test that a person can successfully purchase a product from log in to purchase confirmation.

What do you test well?

Our system is particularly adept at testing form-based UI or functionality that has a predictable input/output. A few other strengths unique to our tool are...
1. ...cases which require multiple users to interact be the same flow be it via email, sms, or instant messages.
2. ...cases built for platforms with constantly changing code and HTML IDs.
3. ...complex flows that need to combine actions like image comparison, audio quality, dragging & dropping all into one.

What don't you test well?

We don't test games, real-time graphs, video content. Stuff like that.

If my application(s) include things you don't test well, why should we work with you?

Even if your business is heavily dependent on any of the few things we do not test well, there is a good chance that you also have a lot of UI that we do test well.
Rather than dedicating testing resources to this type of tedious, repetitive & time-consuming work, you can unload that testing to testRigor and re-deploy your testers to your core testing needs.

Can you also test mobile web, native mobile apps and/or desktop applications?

We can test mobile web & native mobile apps out-of-the-box. We do not currently test desktop applications.

Do you offer an on-premise solution?

Yes, we can offer an on-premise solution.

How are your tests prioritized?

Installing our JavaScript library in your production environment will allow us to cover your most frequently used functionality & flows (up to 1,000 test cases). Secondly, you can also manually create custom tests using plain English or our browser plugin.
Beyond that, tests are run in parallel on our infrastructure, so prioritization is less relevant. We'll execute all tests every time unless you specify differently.

How will I know how much coverage testRigor has given me?

If you are using our JavaScript library & browser plugin, you can be assured that we are covering your most frequently used AND business critical functionality & flows. The best way to mange this is to name & tag these tests with the same naming system you are using in your current test management suite. Test cases are easily searchable.
We also offer a "tree" view that allows you to quickly see what areas of your application are being tested on any given test run.

How many test cases will you cover?

By default up to 1,000. If you need more, just let us know.

What is the minimum & ideal amount of test coverage I need?

It will obviously vary by application, but ideally you want to make sure you have covered your most frequently used functionality & flows AND all of your most critical business test cases.
Where you draw the line here is up to you, but it is unlikely that testRigor will ever be your roadblock. We can automatically discover as many edge cases beyond this as you'd like, but there is a law of diminishing returns beyond 1,000 test cases for most web & mobile applications.

How would you work with our teams that already have automated test suites?

Great question. Most teams have engaged in some level of attempts at automating their tests. For those who have, you've no doubt run into the challenges of creating, maintaining & scaling these tests.
The sad reality is legacy test automation approaches do not scale. Your application development will always outpace your ability to cover new functionality while maintaining flaky test suites. Legacy test automation approaches are a Sisyphean task!
testRigor will get you out of this vicious cycle with tests that mostly build themselves and self-maintain. This reduces your team's level of effort around regression testing so they can be re-deployed to more impactful testing initiatives.

Can you convert the manual test cases I already have?

Yes, we have a browser plugin that allows you to capture a manual test case with the click of a button. Once created, the test will automatically execute and maintain itself, allowing you to move on to more interesting test work.
For complicated test cases, you can use our Reusable Rules & Test Cases interface to create sophisticated tests using plain English commands. No code to learn!
With minimal training, all manual testers can be automation engineers!

Can you really handle truly complex user flows?

Absolutely! You'll be surprised at a how much functionality testRigor will cover automatically. But, of course, there are test cases impossible for any system to execute automatically without a human touch.
We make that process easy & fast (no code to learn) via Natural Language Processing to allow you express in English what needs to be tested.

How do you handle edge case tests?

testRigor will automatically cover frequently used functionality up to 1,000 test cases. If there are specific edge cases you want to discover beyond this, you can quickly create a test by using our browser plugin or modifying existing tests.

How do you handle new features?

New features will be surfaced by testRigor automatically on every new run with a suggested test script. Your team will then verify that this is indeed new functionality and approve/edit the test script. Boom! Test covered.

How do you keep tests stable when I push a new UI change?

This is testRigor's secret sauce!
We are NOT dependent on XPath. Instead, our AI was trained to look at your web & mobile applications the same way a human would. Our tests are designed to emulate the way a manual tester would execute a test.
This approach means that our system can figure out UI changes in a way that conventional solutions are simply unable to do.
testRigor extracts visible elements like texts, inputs and buttons/links and allow you to work with just visible elements based on their type.

Are there any UI frameworks you will not work with?

Nope. Since we emulate a real human running a test, your framework does not matter. We work great with SPA frameworks like React & Angular.

Will tests break if I change development platforms?

Very unlikely. Our clients have changed development platforms and UI frameworks and seen virtually all of their tests survive. We specifically built testRigor with adaptability and stability as our #1 priority.

How would you work with applications which are highly customizable for our varied clientele?

You'll probably want to build several suites; one for common functionality and several for some customer-specific features. This is easy to build and customize in our test creation interface.

How are your tests deployed?

We handle everything on our servers. No need to worry about infrastructure headaches or cost-overruns. Call our APIs from Jenkins, CircleCI or any other CI/CD system you use.

Am I responsible for maintaining the infrastructure required for your tests to run?

We handle everything on our servers. No need to worry about infrastructure headaches or cost-overruns.

How are your reports delivered?

Email, Slack, API, and/or our web app. Totally up to you.

Can your tests be delivered in a framework other than testRigor?

We can easily integrate with test-case management systems like TestRail and Zephyr.

What are custom test cases and reusable rules?

Custom test cases are a way to create a test script using plain English instructions. Reusable rules help our system generate better tests and can be used as subroutines for any test.
You can even streamline your customizations by having custom test cases/reusable rules reference other test cases/rules.

How do you support my team?

Fanatically! All of our plans have ample monthly premium support hours built in. These hours can be used for training or for our assistance in helping you create robust & sophisticated tests.
We designed testRigor to be the easiest test creation & maintenance solution ever, but we are also there via your own Slack channel, email or phone.
We also offer professional services if you are looking for a more holistic solution.

Where do you get test data from?

We generate data based on detected field type (we support about 50 types), and you can provide your own test data as well as allow us to generate random data conforming to a specific format every time we run.

How do you manage to make your tests so stable?

1. We emulate the way a human would interact with the application instead of hard-coded XPaths.
2. We have a sophisticated system which ensures that the page is fully loaded before proceeding.
3. We do reasonable retries for timeouts, browser crashes, etc.
4. Our system emulates the same behavior on the page that you as a human display by interacting only with visible elements (with some exceptions) on the screen or page.

Who builds your tests?

The test creation is completely autonomous and comes from anonymized metadata about how your users are using your applications in production as well as from AI-enabled rules-based engine to cover most important functionality for your app.
We also provide you with a browser plugin to automatically capture tests your manual test team is executing today.
However, you can create/record your tests yourself using our Chrome Extension or just using plain English.