Are Unit Tests a Waste?

Anushree Chatterjee

If you’re someone in the tech industry, even if not a software engineer, then you must have heard about unit testing, seen your team members do it, done it yourself, or are eager to learn to do it.

A popular type of testing, unit testing scrutinizes the very units that build the application. These units could be functions, methods, classes, modules, APIs, and even small, cohesive blocks of code.

The majority swear by the belief that “Cover maximum grounds with unit testing to ensure bug-free code.”

However, James O. Coplien, an Agile coach, is of the unorthodox opinion that most unit tests are waste.

Shocking, right?

You’d wonder why someone with such vast knowledge and experience in the tech world, as this author put down one of the most fundamental forms of testing. It is known to veterans and newbies as the base of software testing, as in the test automation pyramid.

Let’s dig deeper into understanding how our most trusty unit testing, one that we believe to be indispensable in the QA world, can backfire.

Reconsidering Unit Testing

As novel as a concept seems, they all come with their own set of pros and cons. James O Coplien, after years of experience in the industry and seeing its vulnerable side, has compiled an entire document on how unit tests can hinder you. Though the document is long (21 pages, to be exact), let’s go over some of the points that the author makes.

Disclaimer: Keep an open mind!

(especially those staunch practitioners and believers of unit testing)

Let’s begin.

Modern programming methodologies have redefined testing types

Through various experiences and examples, the author lays out how modern techniques like Test-Driven Development (TDD), Object-Oriented Programming (OOP), Agile, and XP have altered the definitions of many testing types, including unit testing.

In the earlier phase of computing (the FORTRAN), when making machines do something for you was new, testing a unit was more straightforward than it is today. A function did exactly what it was meant to, meaning all the logic needed for something to happen was right there. Thus, debugging was easy, and unit testing gave a good idea of the application’s quality.

Now, we are in the age of abstraction and encapsulation, where though a function seems small, it makes multiple internal calls to various other functions to perform a task. Debugging now becomes a task since everything is scattered. This made testing a necessity.

This leads us to the next problem.

Unit tests have gotten more complex and more in number than the actual code

“The test does have a cost: maintenance, computing time, administration, and so forth. “

Modern principles dictate segregating code in a way that makes it difficult to identify a unit in the code. Thus, you might find most unit tests being more complex and lengthy than the actual code. This means higher maintenance costs, even though most of them are unnecessary and do not contribute to the bigger picture.

Unit testing does not equal automated testing

“If you’re going to automate, automate something of value.”

A simple-looking function can turn out to be complex at the time of writing tests if it has loops. In this situation, your unit test will also become complex. As soon as people hear this, they think of delegating this task to automation. But here’s the thing – automation does not guarantee success. It just makes the job easier. What you automate matters.

Automation is a powerful tool, but for repetitive tasks only. For all cognitive purposes, use the human brain!

The green bar fever. Every test should verify a business requirement

Who comes up with unit tests? Usually, developers. In fact, the same developers who wrote the code. These tests do not stem from business requirements but rather from the developer’s fears or fantasies about the application. Tests, unit tests in particular, are used as a metric to define how well the code is written – if all passes and is green, then the code is perfect. This defeats the purpose of learning from testing.

You will see that, in most cases, the reasons behind writing these unit tests could be:

• Paranoia about correctness, which in turn drives out clear thinking and innovation that can actually help in quality assurance
• Lack of analytical skills, test design, or discipline of thinking, thus relying on the machine to do all the work for them
• Difficulties in processes or tools that hinder integrating frequently
• No (or little) understanding of requirements, so they don’t know what to test, thus sticking to their best guess

If developers are incentivized solely based on unit test coverage, they might prioritize quantity over quality. This can lead to a situation where the code has high coverage numbers, but the tests themselves are weak and don’t effectively safeguard against critical issues. Also, writing unit tests for simple, self-evident functions might not provide much benefit. These tests are easy to pass but don’t necessarily catch real-world errors.

Thinking that tests are smarter than code

” Tests don’t improve quality: developers do.”

“Testing does not increase quality; programming and design do. Testing just provides the insights that the team lacked to do a correct design and implementation.”

From these quotes, you might have guessed the gist.

Of the many professionals the author spoke to, he found that most tend to think that if they write better tests, then their code will be safe from bugs. This rather defeats the purpose of writing tests. They are meant to give you more information about your application and tell you what can be improved. In fact, the less is more principle works here. It is not the shinning blade to dub an application as perfect. If developers focused more on understanding the requirements, designing good solutions, and then writing good quality code, they wouldn’t have to depend on testing like a crutch.

Testing cannot replace good development. If you find yourself facing a lot of test failures, then it means that you need to reevaluate your architecture and design regimes.

Unit tests alone cannot give you good-quality code

“Tests should be designed with great care. Business people, rather than programmers, should design most functional tests. Unit tests should be limited to those that can be held up against some “third-party” success criteria.”

If you want to reduce your test code mass, then think before you write your tests. Since the basic principles of code have evolved, making code fragmented and compartmentalized into smaller pieces, just relying on unit testing will not help. You need to utilize other forms of testing more, like integration testing, system testing, exploratory testing, and Monte Carlo techniques. They are more likely to be effective and exercise all those code snippets that unit testing can’t.

Unit tests are great for testing key algorithms for which there is an agreed-upon specification that gives business value. For all other scenarios, if something has business value and you can test it with either a system test or a unit test, use a system test because testing in context is everything.

If you want to reduce test code mass, get rid of unnecessary unit tests

“A smarter approach would reduce the test code mass through formal test design: that is, to do formal boundary-condition checking, more white-box testing, and so forth.”

These are the types of tests the author advises against keeping:

• Tests that never fail. The author states that you’d be better off without such tests.
• Tests that are not impacted by a change in the application under the test’s code. It means that these tests are not checking the functionality of those functions, so whatever they are checking has little to no value.
• Tests that can be assertions in the application code.
• Tests that duplicate what system tests do.
• Tests that target codes that are unreachable through system testing. If your testing interfaces are well-designed and can reproduce the kinds of system behaviors that you see in the real world, then you shouldn’t need explicit unit tests to test dead code. In fact, get rid of this dead code.
• Tests that have zero business value.

The fail-fast culture cannot substitute good-fashioned thinking

“Software engineering research has shown that the most cost-effective places to remove bugs are during the transition from analysis to design, in design itself, and in the disciplines of coding. It’s much easier to avoid putting bugs in than to take them out.”

In the early years of computing, maybe due to scarcer computing resources, a lot of time and energy was invested in designing code, giving rise to well-understood primitives. This led to better relation of functions to business requirements.

However, in today’s fast-paced world, everyone wants immediate results to the point where nobody wants to put in the time and effort to think of good solutions, be it for application code or testing. Even if they want to think, they may not have the time to do so due to the fast-paced nature of modern testing methodologies. This leads to many issues, such as:

• Relying on testers to be the sole judge of whether the test findings are correct. A tester is a third-party when it comes to the application and the test case. If issues do come up, the developer and tester will try their best to ‘fix’ it to take the developer closer to the green bar of a perfect score. This fix usually acts like a bandaid rather than a cure to the underlying problem.
  Inputs from businesses, that is, product owners, when it comes to what should be tested are better since that gives a clear understanding of what business requirements are being safeguarded with testing.
• Developers put half-baked solutions out, hoping that testing will fix it. All this is because of time crunches.
• Relying on machines (like computers with their apps for testing and debugging) to find issues for them rather than doing the thinking by themselves.

Testing is not a metric, but a means to learn about the application

“If the probability of the test passing is 100%, then there is no information — by definition, from information theory.”

Since testing helps pinpoint gaps, many use it as a way to determine if the code is correct or not. But here’s the thing, testing if used to identify right or wrong, will divert the team’s focus from the things that matter the most – an opportunity to learn about the application. A third person’s view on the code is essential, like that of a tester’s, but ultimately what matters is whether the code is solving the business problem at hand and whether the test case is helping ensure this. Read: How to Write Test Cases? (+ Detailed Examples).

“There are two potential goals in testing. One is to use it as a learning tool: to learn more about the program and how it works. The other is as an oracle.”

Unit Testing, but the Right Way

Unit testing when done excessively is not good. But that does not mean that you skip it all together. There are certain areas where unit testing can add value like verifying algorithms or other major functionalities that can directly compromise business requirements.

Here are some tips that will help you come up with unit tests that are not a waste of time.

Way Ahead through Intelligent Testing

We’ve learned so far that unit testing can help you, but only so much. For the rest, you need to rely on testing at other levels. The best way to ensure that your tests run in environments closest to the real world is through system tests, functional tests, and end-to-end tests. For this, we can find a multitude of options in the market. One that shines out in the crowd is testRigor.

Automated testing with testRigor

testRigor is one of the best tools for functional and end-to-end testing. This is because of its awesome and intelligent features. Here are a few of them:

• When writing tests to verify whether business requirements have been met, you’d prefer having your product owners and business analysts contribute to the task. But for them to participate, there needs to be a platform for them to collaborate on. With testRigor, this process becomes very simple since the tool uses generative AI to allow everyone to capture test scripts in plain English language. Thus, the focus can remain on testing rather than on the complications of the platform.
• By harnessing AI, testRigor makes test maintenance negligible. This is a blessing in fast-paced environments where one would rather focus on creative testing.
• testRigor’s rich command library accommodates all kinds of platforms and browsers. It provides a throve of commands that can make complex operations including 2-factor authentication or CAPTCHA, database, file, geolocation, QR code, audio, visual testing, etc., super simple to perform.
• You can expand your testing ecosystem by integrating with most of the popular tools and platforms for test management, CI/CD, infrastructure management, etc.

Here are testRigor’s benefits and documentation, to give you an idea about its powerful capabilities.

To Sum It Up

“Good testing begs skepticism.”

Blindly creating and automating unit tests for every function and its possible values is a poor way to go about testing. No wonder they get a bad reputation for being wasteful. Before diving into creating unit tests, take a pause and think about whether what lies in front of you is worth writing unit tests for or if can it be covered through other tests like system or functional tests.

Always invest your energies and focus in good test design and architecture to avoid having the need to create too many tests.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT