 |
Exploratory testing vs. scripted testing
The rising demand for product quality has led to different testing approaches. Each of these approaches has its benefits and limitations. In this article, let us have a look at two of them, exploratory testing, and scripted testing.
What is exploratory testing?
Exploratory testing relies on the tester’s ability to explore the feature or product to identify bugs. In this context, exploring refers to trying out various test scenarios by considering user journeys. Instead of following a set number of steps, testers are free to get creative and come up with test cases to challenge the system. Ideally, only experienced testers should take up this activity – since they need to have both product knowledge and skills to be creative when designing test cases. This testing approach can seem somewhat unconventional, but it can definitely yield excellent results.
When and why to adopt exploratory testing
This approach mainly comes into the picture when requirements are not transparent or unknown. Since there is an opportunity to freestyle and test different journeys with that feature or product, you’re more likely to catch bugs that would otherwise be overlooked. Testers performing exploratory testing come up with test cases on the fly. Hence, there are no prior documented test cases to be executed.
Having a good understanding of the business problem can come in handy to ensure that no stone is left unturned during testing. With this approach, testers tend to prioritize their testing so that all critical cases or functionalities are checked first. This allows them to give quick feedback to the developers, who can fix the bugs early in the cycle. Since this kind of testing has very few technical blocks, different members of the team, like developers, product managers, and client-facing teams, can be involved. Having such a varied audience can help gain different perspectives, which are necessary for ensuring software quality.
In today’s times, where the focus remains on continuous integration and continuous deployment, being vigilant about the changes being shipped out is necessary. In a rush to deploy to the customer, there is a high chance of compromised quality. Such compromises often lead to edge case bugs that are discovered during the UAT or user acceptance testing phase. This can lead to financial repercussions and also have an impact on the brand’s credibility.
You can read about the types of exploratory testing in greater detail in this article.
When to steer clear of this approach?
If the feature or product is expected to satisfy some key requirements in order to be compliant, maybe due to legal or security reasons, then exploratory testing may not work. Also, if there is a time crunch, then this testing approach may cause delays, especially when experienced testers are unavailable in the team.
Exploratory testing is not as random as it sounds…
As random as it may seem, this approach has a plan of action.
Studying and understanding the bugs that occurred in the previous versions of this product or feature gives a great starting point for identifying the error-prone areas. This helps the testers to prioritize their approach.
Once the approach is finalized, it is necessary to align on what needs to be done. Though the tester has the freedom to get creative with testing, there need to be some ground rules to define the objective of this exercise. Since this is not a very structured approach, being clear about the agenda and having a rough idea helps. Members involved in exploratory testing create a test charter to define a time limit, key areas they intend to cover, and how they plan to test it. This exercise helps identify use cases and different ways of testing them.
Now that the strategizing is done, it comes down to execution. However, this approach is typically time-consuming and hence needs time limits. Timeboxing aids with this problem. The testers allot a time slot for this activity wherein they conduct exploratory testing without interruptions.
After the entire exercise, the observations are recorded and communicated to the respective team for further action.
What is scripted testing?
Scripted testing focuses on having all necessary documents like requirements, test plans, test cases, and test strategy ready before the testing begins. This testing approach works very well in situations where the requirements are clear. Testers can create test cases in advance based on these requirements. Each of these test cases defines the objective of that case, steps to execute, and expected results. This approach is systematic and ensures that all major paths are covered.
Another advantage of this approach is that the documented test cases can be used by other testers or team members. Since the steps for execution and expected outcomes are already defined, experienced testers are not essential to execute them. It is easier to automate these test cases, although technical knowledge may be needed as a prerequisite to work with the automation tools. Since scripted testing is so structured, it is easier to do it in a systematic way within a given time period. Thus making it easier to track ROI and progress.
Some scripted testing limitations
Scripted testing may not be suitable for environments where requirements keep changing frequently. Testers need to invest extra time to prepare the test cases before the feature or product is available for testing. Additionally, this kind of testing tends to limit the tester’s creativity when it comes to challenging the system, thus leading to overlooking some edge case bugs.
Differences between exploratory and scripted testing
Now that you have a good understanding of these two testing approaches, let’s look at the differences between them.
| Scripted Testing | Exploratory Testing |
|---|---|
| Best suited for situations where the requirements are known and do not keep changing. | Best suited for situations where the requirements are unknown, not clear or keep changing frequently. |
| Test cases are prepared in advance before the testing begins. | Test cases are prepared on the fly during the testing activity. |
| Since the test cases are prewritten, any QA person on the team should be able to execute them. | Requires experienced testers for this task since the product knowledge and skill to detect issues is with them. |
| More straightforward to automate these test cases. | Can be challenging and may not be easy to automate these test cases. |
| Test cases are usually prepared by the QA people. | This activity can be done by other members as well like developers, product managers, client-facing analysts – to gain more perspective. |
| Progress and time the operation will take can be predicted. | Due to the absence of concrete steps, this approach can be time-consuming and usually needs time boxing. |
| Edge case bugs tend to get overlooked since the tester sticks to the prewritten test cases. | Generally works better at catching edge case bugs since the tester is free to explore the system and try out different user journeys. |
| Since there are predefined test cases with expected results, features and bugs are not mistaken for one another. |
Due to the absence of a predefined, elaborate requirement, there is a chance that a bug may be mistaken for a feature (and the other way around!). |
| Finding critical bugs earlier in the cycle may be a challenge due to following the execution sequence of test cases. |
Since the tester decides which areas need to be tested first, critical bugs can be discovered and fixed sooner in the testing cycle. |
| It is easy to map which features have been tested. | Due to the unstructured nature of this approach, it is difficult to map which feature has been tested. |
| Due to the systematic nature of this approach, it is easier to maintain test cases and their results for all historical features and changes in the system. |
Maintaining a historical record of what changes or features have been tested through this approach is much harder. |
Using both approaches to ensure better quality
From the above table, we can see that both approaches have their pros and cons. It is beneficial to balance the testing scope between the two to ensure better and more efficient test coverage.
Using scripted testing to check all the primary test cases that cover necessary features from a compliance standpoint is helpful. In fact, this can then be delegated to other testers who may not be experienced enough or who do not have the domain knowledge. This frees up the more senior testers to take up exploratory testing to ensure that all those edge cases and user journeys are checked.
Using automation for scripted testing
Since many scripted test suites have to be executed repeatedly, automation is the only answer in most cases. Smoke and regression test suites are usually the first candidates for automation, since these tests need to be run all of the time. And as you might already know, testRigor is a great codeless solution for any functional end-to-end scripted tests – both in terms of test creation and test maintenance. With testRigor, the whole QA team can participate in test automation (and not just QA engineers and SDETs), making higher quality more feasible for the company.
Conclusion
Be it exploratory testing or scripted testing, both have their strong points as well as limitations. It is essential to understand what kind of environment the testing is happening in. Based on those requirements, these two approaches can be customized and applied to ensure better quality of the end product.
