Turn your manual testers into automation experts! Request a DemoStart testRigor Free

Dollar Escape Rate. How to Calculate ROI for QA Automation

In Software Quality Assurance it is often hard to estimate the value that QA is bringing to the table. The impact of QA is so broad that it is really challenging to gather it all together and assign a dollar value to it. However, we can attempt to assign a value to QA provided that we will be able to enlist help from various parts of the organization.

How to calculate the price of a bug?

Let’s get to the basics.

How does QA help? It helps to prevent issues from escaping to the end product that is handed to the customers.

How do we measure issues? By bug tickets and specifically the ones that had been found in production and/or affected customer(s).

It is straightforward to find the list of issues like this in an issue-tracking system. Moreover, this kind of metric is widely known already as Defect Escape Rate.

That’s great, but do we assign a dollar value to it?

This is where we need to partner with people in Finance/Sales/etc. We can ask those people to provide us with data on how much business has been lost/not acquired within the last 12 months.

Once you have the data for the amount of business lost in the last 12 months (let’s call it “A”) and have the number of defects total which escaped to production in the last 12 months (let’s call it “N”), then you can calculate the cost of a bug escaped to production as:

Cost of a bug in production (C) = A/N = Amount of business lost divided by the number of issues escaped to production.

The original amount of money A which was lost due to bugs last 12 months we would call Dollar Escape Rate.

Next is analyzing why these issues escaped production. What was the main reason why they escaped? Was it because there was not enough bandwidth to do proper regression testing before each change was released?

If so then you can estimate how many tests are being performed on every change which gets to production. Let’s say, on average we can validate (let’s call it “M”) scenarios before releasing software to production.

Therefore, you can very roughly estimate that doubling the number of validated scenarios (M) might decrease the number of issues escaped to production (N) by 2.

Since we know the Cost of a bug in production (A/N), we can estimate that the amount of business lost will also be decreased by 2.

ROI Formula

This way we just calculated the ROI of doubling the number of tests which is A/2 or half of the amount of business lost to bugs in the last 12 months.

In general, the formula will look something like this:

ROI = A * M2 / (M1 + M2)

Where:

A = Amount of business lost in last 12 months due to issues escaped to production (the Dollar Escape Rate)

M2 = The number of additional new scenarios that will be validated with the new change

M1 = The number of scenarios validated currently

For example, if the amount of business lost last 12 months was $1,000,000 and we were validating 1,000 scenarios on every change on average, then adding additional 3,000 scenarios to validate will give us:

ROI = $1,000,000 * 3,000 / (1,000 + 3,000) = $750,000

Therefore any total budget below that amount would be a reasonable investment.

Conclusion

This ROI formula, in particular, helps to partially justify investment into test automation. Since the number of scenarios validated with manual testing is very low, automating tests to allow validating more scenarios on every change should increase the coverage dramatically, therefore delivering high ROI.

And, we believe testRigor is by far the fastest and easiest way to build end-to-end regression tests since your existing workforce (Manual QA, QA Automation Engineers, Product Managers, Business Analysts) will be able to do it about 15 times faster than traditional automation with Selenium.

Don’t believe us? See how it helped other companies here:

or talk to us here.

You're 15 Minutes Away From Automated Test Maintenance and Fewer Bugs in Production
Simply fill out your information and create your first test suite in seconds, with AI to help you do it easily and quickly.
Achieve More Than 90% Test Automation
Step by Step Walkthroughs and Help
14 Day Free Trial, Cancel Anytime
“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”
Keith Powe VP Of Engineering - IDT
Related Articles

Visual Regression Testing

Imagine a popular e-commerce website that recently underwent a design update. While the core functionality remains intact, a ...

Paid Automation Testing Tools: Not All Created Equal

As software testing trends continue to progress, codeless automation tools have gained traction among teams looking for faster ...
On our website, we utilize cookies to ensure that your browsing experience is tailored to your preferences and needs. By clicking "Accept," you agree to the use of all cookies. Learn more.
Cookie settings
Privacy Overview
This site utilizes cookies to enhance your browsing experience. Among these, essential cookies are stored on your browser as they are necessary for ...
Read more
Strictly Necessary CookiesAlways Enabled
Essential cookies are crucial for the proper functioning and security of the website.
Non-NecessaryEnabled
Cookies that are not essential for the website's functionality but are employed to gather additional data. You can choose to opt out by using this toggle switch. These cookies gather data for analytics and performance tracking purposes.