Building Test Automation Framework is a Stupid Idea

Artem Golubev

The Test Automation Dillemma

I recently had a conversation with a company and asked them about the numbers of their test automation efforts.

They said that having started two years ago, they hired an ‘N’ number of QA automation engineers in the process. Diving deep and out of curiosity, I asked how many automation tests they had built in those two years. To my utter surprise, it appeared that their one QA automation engineer, on average, has built 5 test cases per year.

No, this is not a typo. Five. Per year. Per person.

Admittedly, this is significantly below average.

Well, the issue was not just that they had spent an enormous amount of time building a test automation framework. But another hindrance is in the fact that the product was constantly changing, often drastically. Therefore, their test automation framework had to be updated once every two weeks. And here I’m not even talking about the tests themselves, this massive test maintenance effort only to maintain the test framework. Then, tests are separately handled during maintenance activities.

Can you determine the vast time, effort, and money wasted here? Frustration and boredom of engineers are complementary in such situations since their skills are underutilized.

Why Reinvent Wheels and Bicycles?

There are around 26 Million e-commerce sites out there, thriving and evolving. At any given moment, tens of thousands are updating/improving their checkout flow to improve UX. Their QA engineers are writing the same checkout flow test with very little difference between them. All ten thousand of them!

It is an all-too-common problem that leads to wasted effort, inefficiency, and a sense of reinventing the wheel. I know you can clearly see what is happening here and might be thinking: “Is there a way to leverage the common knowledge base just to get the tests done somehow, bypassing all this loss?”.

Well, it is possible now with testRigor’s Generative AI. Check this video for information.

The Story of Page Object Straitjacket

Page object came into the picture with good intentions, but it is probably the worst invention in the test automation landscape. Well, at least the way people usually use it.

Firstly, incorporating POM(Page Object Model) in your test automation means writing a lot of additional programming code. You have to put all this hard effort into doing that instead of your framework taking care of it automatically.

Secondly, imagine you have successfully built your page objects for all your pages with XPaths figured out in their best possible form. Now suddenly, BOOM! A user action got moved to another page of the website. Can it get worse than this? All your precious time and efforts got wasted in one shot. Not to mention, you’ll have to disheartenly redo this same action from scratch on another page object using, probably, another XPath.

You can see here even though the initial intention of creating the page objects was good. But in reality, it puts unnecessary maintenance and coding overhead, bringing very little value to your overall test automation process.

Sisyphean Test Automation Labor

Now, let’s get straight to the point of the test automation framework itself. This process brings a gigantic Titanic overhead, where you need to configure, set up, and write code(!) for tons of supporting aspects such as:

• Build Management
• Configuration Management
• Logs Generation
• Reporting Support
• Utils
• Page Objects

This above list is not at all about building the actual tests for the Application Under Test(AUT). This list is for the setup you require every time you start a new project. Apparently, this setup has a really high price tag.

Now, let’s look into the “Test Automation Framework” itself and what we are trying to achieve with it. We are literally filling up page objects with related methods to perform actions on software/applications. But that’s the fallacy we discussed in the previous section – it is the wrong model for achieving efficient QA.

Unfortunately, eventually you will bear the price of constant updates just because of using OOP incorrectly. The correct model is based on the application’s functions, NOT the pages. Ideally, even if the pages change, functions should stay as is. But right now, you are hardcoding business logic in your tests based on the details of implementation (XPaths, CSS selectors, locator IDs, etc.). This whole exercise will cost you dearly if you sometimes need to do things differently. The amount of code and complexity will blow up, making it extremely difficult for your colleagues and team members to deal with the framework.

Think about when you last joined a company, and they shared a test automation framework with you to work on. How much time did it take you to adapt? Or did you re-write it again from scratch? Your response has a hidden answer regarding the unwanted labor and complexity of programming a test automation framework.

We have a Better Solution

I know there is a better way to deal with all test automation issues discussed above, and the cherry on the cake is that it is achievable just through plain English.

Just imagine that your test cases, which you wrote anyway, are now executable out of the box. You can use your organization’s terminology in plain English to describe user actions, workflows, navigation, etc., and there is no use of XPaths/CSS selectors/IDs, which constantly change and cause instability of test scripts.

Isn’t it exciting? Weeks, months of effort, and cost are saved. Yes, it is possible right now, and here is an example of how to do it, have a look!

Conclusion

When the whole world is going superlatively enthusiastic over generative AI and its uses, test automation is not an exception. Following the old-school test automation path and making the same mistakes is not fruitful for anyone – customers, organizations, final product, QA engineers, and developers.

The need of the hour is to use the most innovative technology available and adopt a more efficient way to do QA. How else will you be able to meet the dynamic and fast DevOps and Agile requirements? Intelligent test automation is the only answer. Leverage the power of AI to grow faster, with minimum efforts and cost, leading to a faster time-to-market and excellent product quality.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT