Test Estimation Techniques: The Backbone of Your QA Strategy

Anushree Chatterjee

Do you remember the healthcare.gov website launch in the United States in 2013?

Within 2 hours of its launch, significant issues like slow load times, difficulty in user registration, and frequent crashes started taking place as almost 250,000 users, five times more than the anticipated load, tried to avail themselves of the benefits through this website.

The problems were partly attributed to inadequate testing and poor estimation of the testing effort required. The site was reportedly tested for only a short period before its launch, and the testing did not fully simulate real-world conditions of high traffic and diverse user actions.

This is an example where test estimation could have significantly impacted the website’s quality.

What is test estimation?

A software application can be deemed complete only if it has been developed and thoroughly tested. Test estimation is the process of predicting the amount of time, effort, and resources that will be required to carry out testing activities in a software development project.

In fact, you can get away with developing just a few basic features, but you cannot do away with intensive testing of all the developed features. For example, you can choose to build a very basic bicycle and not add other features to it, like gears, lights, or a basket to carry things. Despite the product’s simplicity, it needs to be tested thoroughly to ensure that it performs as expected in the real world.

Aspects of test estimation

Here are some key aspects of test estimation that you should consider.

• Scope of testing: You need to know what should be tested, including functionality, performance, security, and usability, and determine the depth of the testing based on project requirements and risks.
• Resource allocation: Estimate the number of testers needed and assess the skillset that should be present with these testers.
• Time estimation: Determine how long each testing phase in STLC will take, including time for planning, designing, creating, executing, and evaluating tests, as well as retesting and regression testing.
• Tool and infrastructure requirements: Identify the need for testing tools, like automated and performance testing tools. Here are the top 7 test automation tools.
• Risk analysis: Consider potential risks and incorporate buffer time for unexpected issues. Read here a good article about risk-based testing.
• Historical data analysis: Utilize data and metrics from past projects to inform estimates.

Why should you perform test estimation?

You will benefit from performing test estimation before jumping into the project head-on. Here is how it will help you.

What should you estimate in test estimation?

Various aspects need to be thought over during the test estimation process. Here is a list of them.

Types of test estimation techniques

Now that you’ve understood what test estimation is and how it can benefit you let’s delve into the different test estimation techniques.

Work breakdown structure (WBS)

WBS is breaking down the testing project into smaller, more manageable components. Each component is then estimated separately, and the estimates are summed up to determine the overall effort.

Here’s an example. In a project to develop an e-commerce website, the WBS might break the testing into components like UI testing, database testing, performance testing, security testing, and so on. If UI testing is estimated to take 40 hours, database testing 30 hours, and the other components similarly, the sum of these gives the total testing effort.

Three-point software estimation test

This technique involves calculating an optimistic (O), most likely (M), and pessimistic (P) estimate for each task. The final estimate is often calculated using the formula.

Three-point Estimate = (O + 4M + P) / 6

Where O = optimistic estimate, M = most likely estimate, P = pessimistic estimate

Here’s an example. Assume that the optimistic estimate for a particular test is 10 hours, the most likely is 15 hours, and the pessimistic is 25 hours.

Three-point Estimate = (10 + (4*15) + 25) / 6 = 16.67 hours.

Wideband Delphi method

This is a consensus-based technique where a group of experts provides estimates anonymously. The estimates are discussed in multiple rounds until a consensus is reached. It comprises the following process and steps:

1. Selecting experts who can provide relevant contributions
2. Kick-off meeting to explain the process, discuss the scope and objectives, and provide any necessary background information
3. Initial estimation round, usually done in writing and with anonymity
4. Compilation of estimates
5. Discussion on the estimates
6. Iterative re-estimation
7. Final estimation

Here’s an example. A team of five testers provides anonymous estimates for testing a module. They estimate 20, 22, 18, 25, and 21 hours. After discussion and multiple estimation rounds, they agreed on a consensus estimate of 22 hours.

Use-case methodologies

This approach bases the estimation on the application’s use cases. Use cases are a set of actions or event steps defining the interactions between a role, also known as an ‘actor‘, and a system to achieve a goal. This method helps understand and capture functional requirements. Each use case is analyzed for its complexity and the effort required. It comprises the following process.

1. Identifying use cases
2. Classifying the use cases based on simplicity
3. Assigning effort estimates to each of them
4. Calculate the total effort required
5. Consider reusability and non-functional requirements
6. Review and adjust

Here’s an example. If an application has ten simple, five moderate, and two complex use cases, and if simple use cases take 3 hours, moderate ones 6 hours, and complex ones take 15 hours, the total estimate would be (10*3) + (5*6) + (2*15) = 90 hours.

Distribution in percentage

This method is relatively straightforward as it estimates testing effort as a percentage of the overall effort required for the project, typically the development effort. This method is often used due to its simplicity. It is based on historical data or industry standards that indicate how much time, on average, is spent on testing in relation to development.

Here’s an example. If historical data suggests that testing takes about 30% of the development time and the development time for a project is estimated at 300 hours, the testing estimation would be 30% of 300, which is 90 hours.

Method of ad-hoc

This technique is a less structured and more informal approach compared to other systematic estimation techniques. It relies heavily on the personal judgment, experience, and intuition of the individuals involved, typically seasoned professionals who deeply understand the project requirements and the testing process.

Here’s an example. An experienced tester might estimate that testing a particular feature will take approximately two days based on their past experience and understanding of the feature’s complexity.

Functional point analysis

This technique involves quantifying the functionalities of the software to be tested. Each function point is assigned a weight based on its complexity, and the total weight gives an estimate of the testing effort.

Here’s an example. An application has 20 simple function points, 30 average, and ten complex if simple points are weighted 1, average 2, and complex 3.

Total Weight = (20*1) + (30*2) + (10*3) = 110, which then can be translated into effort based on historical data.

Conclusion

Test estimation is not just a step in the QA process but a strategic tool that will aid you in project management to ensure the delivery of quality products. Based on your project and team, you can opt for one or a combination of the abovementioned techniques to devise your test estimates.

You can save time using intelligent testing tools such as testRigor that automate various aspects of the testing process. This will help you cut down on time and deliver high-quality products quickly, keeping your customers satisfied. Moreover, if you’ve chosen a test automation tool wisely, processes like test case creation, execution, and maintenance will not plague your team and deplete your budget.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT