7 Features of a Good Automated Test

Artem Golubev

If you think of it there are 7 features a proper automated test should follow:

1. The test can be executed programmatically
2. The test is repeatable. If the application didn’t change, the test result doesn’t change
3. The test ends with a validation
4. The test is stable enough to be used in CI/CD
5. The test is very easy to understand
6. The test requires as little maintenance as possible
7. The test is independent and can run in parallel with all other tests

Let’s dive into these points:

The test can be executed programmatically

This just means it is an automated test and not a manual test, we want to distinguish here for clarity

The test is repeatable. If the application didn’t change, the test result doesn’t change

This is basics related to unique test data generation. For example, a test of a sign-up flow. If you were to try to write an end-to-end test for production testing sign-up flow you can be sure your test won’t work the second time with the same email if it requires a unique email to register.

The test ends with a validation

This is a best practice. Think about it, if a test doesn’t end with a validation then you don’t know if the last action succeeded or not. Of course, we are not talking about the case where we need to perform some tear-down actions just about the test logic itself.

The test is stable enough to be used in CI/CD

If a test flakes out, it is not good enough for CI/CD which is the goal for most companies. This makes the test almost useless most of the time since it can’t be used in a CI/CD context.

The test is very easy to understand

We rarely create tests alone in a vacuum, our colleagues need to understand what is going on to be able to adapt existing tests to new functionality. Even if you create tests alone, after some time you might start to forget what was going on in the test and it would look unfamiliar to you.

The test requires as little maintenance as possible

This is really straightforward – the less time we spend on test maintenance the more time we can devote to something more meaningful, like writing new tests. Therefore, anything that helps to reduce maintenance time is helpful.

The test is independent and can run in parallel with all other tests

At some point, especially with end-to-end tests, any company will run into situations where tests take too long and slow down the release cycle. This is where the first thing that comes to mind is to run tests in parallel for speed. If tests were written in such a way that they will run independently it helps to simplify this process by making it a DevOps effort and not a rewrite effort.

How to achieve it as easily as possible?

Introducing testRigor, where you can write tests in plain English from the end-users perspective which would simplify your life greatly and assist with all the features above:

1. The test can be executed programmatically – clearly, testRigor is a test automation system, so it does help. Moreover, it empowers people who are not engineers to create test automation.
2. The test is repeatable and if the application didn’t change, the test result doesn’t change. Here testRigor makes it trivial to generate unique data.
3. The test ends with a validation. Validations are made easy in testRigor too.
4. The test is stable enough to be used in CI/CD. A unique feature of testRigor is that the tests are as stable as if you executed them manually. Moreover, testRigor is built for CI/CD and easily integrated with Jenkins, CircleCI, GitHub Actions, GitLab CI, or anything else you can think of.
5. The test is very easy to understand. Can’t be more simple compared to the plain English of testRigor.
6. The test requires as little maintenance as possible. Tests on testRigor only need to be changed when your executable specification changes.
7. The test is independent and can run in parallel with all other tests. This is a built-in feature of testRigor, all tests by default will run in parallel.

Curious to try it out? testRigor has a forever free open source version here.

Want to talk to someone and learn if it will work for your system? Request a demo here.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT