SmartBear’s TestComplete Alternative

To automate or not to automate your tests is not a question anymore. The question is how to build automation tests the right way, as well as which tool to choose – so that it aligns best with your needs, the company’s budget, and your team’s capabilities.

The ideal scenario for nearly any company would be to have a single testing framework for all the products the company has, and the whole QA team participating in writing and maintaining both manual and automated tests. The latter can be especially tricky to achieve for the following reasons:

• Most automation tools typically require strong coding knowledge and are designed for automation engineers. Even if they do have record & replay feature – in most cases you’ll find them to be able to only cover basic scenarios
• Not everyone on the QA team typically has the required automation skills due to the company’s budget constraints
• There’s a common trend in the industry that automation engineers aren’t too fond of any type of manual testing

All these reasons typically lead to automation QA working independently in their own framework – which results in a disconnect with manual QA, and the whole testing process not being as efficient as possible. Luckily, nowadays there are companies that understand these challenges and strive to create a perfect tool that enables the whole QA team to participate in the automation process. Today we’ll look into two most powerful and popular options, and hopefully give you a clear picture of their advantages and limitations.

Why compare testRigor and TestComplete?

There are a lot of similarities between the two in terms of the supported platforms and types of testing – both provide a broad coverage and strive to be the only QA automation tool you’ll need. There also is a massive difference. TestComplete is a tool built primarily for automation engineers, but additionally has a rather strong record & replay feature that enables less technical team members to contribute. testRigor on the other hand is built specifically for manual QA – and allows you to create complex automated tests with just plain English.

Platform and types of testing support

• Both support Desktop Web, Native, and Hybrid Mobile applications. TestComplete also supports Native Desktop applications
• Cross-browser and cross-platform testing is supported by both
• Both tools have integrations with device farms for mobile testing – BitBar with TestComplete and BrowserStack with testRigor. Also support emulators/simulators
• API testing is supported as well (but costs extra in TestComplete)
• Load testing is supported by both. TestComplete has integrations with LoadComplete and LoadNinja which cost extra, while testRigor has load testing built-in

Test creation

Both here have an option to write tests from scratch, as well as use a test recorder. It’s worth noting that TestComplete’s recorder is considerably more advanced than many other recorders out there – but is still not as good as the one testRigor has. As an example, we’ve recorded a test on the amazon.com website with both tools. While TestComplete got confused with some of the dynamic elements on the page as well as with some validations, testRigor completed the job without a hiccup. Results also depend on the browser you’re using, for example when using Chrome instead of Firefox, TestComplete still failed the pre-recorded test but with fewer issues.

Overall though, both tools are powerful and are capable of creating complex automated end-to-end tests; but it’s important to keep in mind that TestComplete was designed for automation engineers and always requires oversight and guidance from them.

Here’s how clean a sample automated test looks like in testRigor.

click "Best Sellers"
click "Books" above the "Camera & Photo Products"
click "Keep Sharp:Build a Better Brain at Any Age"
enter "1" into "Qty:" near the "1"
click "Add to Cart" above the "Buy Now"
check that page contains "Added to Cart"

Test maintenance

We all know that test maintenance can easily become a rather grim daily task, taking a lot of time that would have otherwise been spent on developing new tests. This usually happens if the test structure is not built the right way, or if the tests themselves are flaky (due to test structure or framework inefficiency). Both tools here strive to simplify this process in order to reduce time and effort. Both are very stable and very reliable, but you’ll notice that with testRigor test maintenance is so simple it takes virtually no time.

Continuous integration

Both TestCompete and testRigor here do a great job and allow for integration with virtually any CI/CD tool on the market. testRigor simply has a bash script, while TestComplete does it through either command line, COM interfaces or a special utility.

Dashboards and reporting

Both tools are great here, with detailed reporting dashboards including screenshots indicating where exactly the test case failed.

Pricing

TestComplete is certainly one of the most expensive test automation tools on the market. For example, the license cost for a team of 4 people will be around $31.000 initially, along with $7000 maintenance per year. The price will double in case you need 8 licenses, and so on. However, a lot of services such as AI object recognition, API testing, cross-browser testing, cloud testing, etc. come at an extra cost.

testRigor is more affordable, with the difference in the pricing being per company and number of test suites, which won’t increase in case the number of users expands. Price starts at $1.800/mo per 2 test suites and unlimited number of users. You’ll notice an even bigger difference once you factor in salary costs for automation vs manual QA.

	TestComplete
Created for	Primarily automation QA, QA analysts/testers can help with some tests too	Primarily QA analysts/testers – can create complex tests with just plain English
Test creation	Supports scriptless, Python, JavaScript, VBScript. Tests are very stable and very reliable. Interface is a bit dated and sometimes complicated, but gets the job done.	No programming skills required. Complex algorithms behind the scenes do the hard work for you. Tests are very stable and very reliable. Modern simplistic interface.
Ease of test maintenance	Medium	Easy
Platforms supported	Web, Native Desktop, API, iOS, Android, Mobile Web.Cross-browser and cross-platform testing supported.	Web, API, iOS, Android, Mobile Web. Cross-browser and cross-platform testing supported.
Pricing	Very expensive. Perpetual license per machine(user) + regular 23% maintenance costs + add-ons cost extra	More affordable. Price is per number of test suites; supports unlimited number of users
Free tier	30-day free trial	Unlimited free version (tests will have public visibility)

Conclusion

We think that both tools are certainly very powerful in terms of both features and capabilities, and are able to help you build a robust test automation coverage. TestComplete has a slight edge here with regard to settings granularity, while testRigor wins hands down in terms of overall interface simplicity, and, most importantly, ease of both test creation and test maintenance even for complex applications.