Black-box vs Gray-box vs White-box Testing: A Holistic Look

Artem Golubev

Software testing defines whether the application meets the stakeholder requirements. There are multiple popular approaches to breaking down testing into categories. One of them is Black box vs. Gray box vs. White box testing. 

Before learning the differences between them, let’s first define and understand them individually.

What Is Black Box Testing?

Black box testing is a software testing technique that examines the functionality of a system without knowing its internal working or structure. Imagine that the box is your software, and you interact with it without opening the box. You don’t know what’s inside.

This method can be applied to various levels and types of testing – acceptance, integration, end-to-end, etc.

• Equivalence Partitioning – the idea here is to partition the input domain into several equivalence classes such that a test case could be written for only one element from each category.
  For example, consider testing a password text field where the user can enter the password in the range of 6 to 10 characters. We can partition this set into 3 equivalence classes:
  • < 6 characters – test should fail
  • >= 6 and <= 10 characters – test should pass
  • > 10 characters – test should fail Instead of testing the whole class, we tend to focus on one value from each group. So for example, 4 characters for the first one, 8 for the second, and 12 for the third.
• Boundary Value analysis – boundaries are where defects tend to occur, so we define test cases for boundary values of the input domain to improve the defect coverage and testing efficiency. Generally, the boundary value contains minValue, minValue + 1, maxValue, and maxValue +1. For example – 6 is the minValue, and 10 is the maxValue for the above password field testing,
• Compatibility testing – this type of black-box testing is done to ensure the product compatibility with different computing environments – operating systems, device processors, machine architecture (32-bit or 64-bit), etc.
• Error guessing – requires skilled and experienced testers to design test cases for error-prone functionality. Examples include testing for null pointer exceptions, form submission without entering data, etc.

Black box testing is sometimes considered to be the least sophisticated of all three. However, it cannot be further from the truth. It is the one that is the closest to the perspective of your end-users, who don’t have a clue about the underlying code structure. Thus, you shouldn’t think of black box testing as being “better” or “worse” than other types, rather it is that each type best serves for specific needs.

What Is White Box Testing?

White box testing is a software testing technique that is… you guessed it – the opposite of black-box testing. You do know what’s inside the box this time. White box testing encompasses testing from the development perspective, with the knowledge of the underlying code. Thus, the white box is ideally suited for low-level testing, including unit and integration testing. The above is also the reason why a good chunk of white-box testing is done by developers.

1. Mutation (or changes) testing – Works by making small radical changes to the code and seeing if the tests still pass to check for the robustness of the code.
2. Integration testing – Multiple components of the application are tested to see if they work well in conjunction. Integration testing might include internal as well as external code. Here’s an example of an external integration: the Slack chat component combined with various products such as Zoom, Calendar, Jira, etc. We can verify if these integrations work fine for boundary cases.
3. Paths and Data-Flow testing (DFT) – Path testing looks for broken or redundant conditional logic. In DFT, variables passed throughout the code are tracked to look for un-initialized and poorly allocated variables.

1. Path Coverage – In the path-coverage approach, engineers write unit tests for all the possible paths for the program’s code control flow and assert for redundant or inefficient paths.
2. Branch Coverage – In the branch-coverage approach, we first look for all conditional and unconditional branches in the code, and then unit tests are written for them.

if (expression X is True)
  Run code A
  Run code B
else
  Run code C

Here, code A and C are conditional branches, and code B is an unconditional branch as it will run even if the expression X comes out to be false.

What Is Gray Box Testing

So we’ve just learned that in black-box testing, testers are unaware of the internal code structure. In white-box testing, we have full transparency over the application code under test. Gray-box testing combines the black-box and white-box testing benefits, where testers are approaching software from an end-user perspective, while having partial knowledge of the code internals. It is commonly used for end-to-end testing, regression testing, pattern testing, etc.

Rather than going by functionally testing every path in white-box testing, testers go by focusing on paths that affect the end-user functionality. Gray box testing works by combining inputs from the above two techniques – by performing white-box testing using static application code and black-box testing by discovering vulnerabilities for a possible attack from the end-users perspective.

Conclusion

• Each testing type is best suited for its purpose, and it is common to have multiple testing types used for the same project.
• All of these testing types can be automated. White-box testing is common to be done by developers, while black-box and gray-box are most common to be done by the QA team.
• White-box tests are typically smaller by their nature, faster to execute and produce results.
• It is ideal to have black-box end-to-end UI test coverage at the end of the testing cycle since it will give you a whole picture of the project as seen by an actual human user.
• If you’re familiar with testRigor, you might already know that it makes UI end-to-end test automation dramatically more approachable compared to any other test automation tool. And if you haven’t tried us out yet, you might want to register a free account – and be amazed at how simple it is to create comprehensive end-to-end UI tests.