Why Cucumber and SpecFlow Died?

Anushree Chatterjee

Cucumber and SpecFlow are both popular in Behavior-driven Development (BDD). Yet both have been abandoned. While this news has brought up visceral feelings in the testing community, some are happy that the end is nigh, others anxious about their present setups using these tools, the question that begs to be asked is, “Why did this happen?”

Let’s understand why these two tools are becoming relics of yesteryears, with management across companies eager to switch to simpler implementations. When you understand the root of the problem, you’ll be better positioned to craft your software development process.

The Eternal Conflict

Behind every product, developers and product owners have a power struggle (whether visible or not).

Why?

Because the former thinks that without good code, the application is a failure, while the latter thinks that without the proper specifications, even the best and most well-written code is a waste of time.

Let’s look at this from a product owner’s perspective.

Why do we build applications? To solve real-world problems. It is immaterial to your customer whether you wrote the application in Java, Python, Ruby, or with special sands from Saturn. As long as it works exactly the way they want it to, you have their blessing (loyalty, money, business). Hence, product owners want to make sure that the application “behaves” as their users need it to. This also means that the application is solving the user’s problem.

So, if these guys are asked what they want from QA, it will always be to ensure that the application behaves as expected.

Now, developers also look at the application’s behavior, but from a different perspective. They are more worried about the code because, let’s face it, it is the code that makes the application “behave”. So naturally, QA for a developer will be to ensure that the code is working properly.

While both these viewpoints seem sort of intermingled, they differ when it comes to software testing. One party wants test cases to check the code – its output, health, and more – which brings in unit testing and integration testing. The other party wants to take a broader approach and test the behavior of the application – its performance, functionalities – that is, testing types like end-to-end testing, user acceptance testing, and the like.

Testers get caught between these two parties, wondering what kind of testing they should focus on. Well, if you think about it, testers act as the gatekeepers to ensure that the product is as expected. But unless they know what is actually expected, it becomes hard to guarantee a satisfying product. Unfortunately, this type of confusion is quite common within teams and often leads to misunderstandings and misaligned interests.

In fact, these conflicts of interest ensured that the communication gaps between the 3 Amigos remained, which in turn affected product quality. Who suffers amidst all this? The end user, due to bugs or unwanted system behavior.

If you zoom out of these internal politics and think from a rational perspective, wouldn’t you agree that without knowing how the application should behave, no party (developer, product owner, and even tester) can do their job?

That’s why “behavior” is so important – it affects everyone.

Now, the next question: how should one communicate this behavior? Should it be in code? Should it be a requirement? If the 3 Amigos need to agree on a single path ahead, then the mode of communication needs to be one that is acceptable and understandable to all.

Enter, BDD.

The True Essence of BDD

If you objectively look at the essence of BDD, it is a beautiful thought. It is more than mere code or test cases.

BDD is a process really that includes a set of practices that help different team members collaborate and ensure that the end product is as the user expects it to be. The term “behavior-driven development” isn’t just about coding. It talks about the complete act of developing a product – from ideation to release.

In Agile environments, keeping up with product quality can be challenging. Knowing the expected behavior of the application is helpful for the 3 Amigos (developers, testers, product owners) as it helps each do their part to achieve this expected outcome. So, if we simplify BDD, we want to do the following:

• Have all stakeholders on the same page right from the beginning
• Use a common language that is understood by all to define the system’s behavior before everyone gets busy with their tasks
• Have a common language that can be used to test both the code and the behavior of the application
• Try to use these defined behaviors to automate testing

You get the gist of BDD, right? – Focus on the bigger picture. BDD is about behavior and collaboration, and automated testing is a byproduct of that. This means teams need to come out of their silos and work together, use collaborative tools, and achieve a common goal – the expected behavior of the application.

The Golden Run of Cucumber and SpecFlow

In the 2000s, BDD came into being as a response to TDD (Test-driven Development). The abstract thoughts of BDD were made tangible through test frameworks. Two popular ones are Cucumber and SpecFlow. The IT industry was so impressed with this new way of thinking that these BDD frameworks quickly became everyone’s top choice for test automation. These tools also started supporting different programming languages and were integrated with multiple other tools like Selenium to make the behavior-driven approach a reality. In fact, these two tools found impressive godparents to sustain them – SmartBear took up Cucumber, whereas Tricentis took up SpecFlow.

Of the many brilliant themes within BDD, the biggest novelty is probably the “common language” that is shared between different stakeholders. In the tangible version of BDD (test frameworks), it is the Gherkin language. It is a syntax that uses plain English to define behaviors in the Given-When-Then format. When everyone speaks the same language, it becomes easier to collaborate on defining requirements and acceptance criteria. Since almost everyone used some or the other BDD framework back then, Gherkin became famous as the one thing that made BDD a reality.

The Downfall of Cucumber and SpecFlow

All was going well until reality caught up with BDD. Though the premise of BDD is great, its implementation was heavily realized through Gherkin and test automation tools. While Gherkin looks simple, it is quite challenging to keep up with. Product Managers and Business Analysts found it too quirky to learn and use. Cucumber and SpecFlow became accessories, an additional and complicated layer of abstraction. A lot of Directors of QA, when implementing Cucumber or SpecFlow, saw nothing but unnecessary overhead on automation that they were already doing.

To sum it up:

• BDD started getting correlated to automated testing. This clearly reduced the essence of BDD to just mere test automation being done in a complicated manner. As test suites grew, these step definitions often became complex, with many layers of abstraction and interdependencies. Ultimately, Cucumber and SpecFlow ended up getting the rap for slowing down test automation performance.
• Gherkin, which is wrongfully synonymous with BDD, didn’t work well when you had to write complex test scenarios. It became a grammar exercise rather than an easy way to define system behavior. Product Managers and BAs who once thought that BDD would finally simplify their efforts to participate in the test automation process found the syntax taxing. Convincing all teams to participate in the process started getting difficult because of that.
• While tools like Cucumber and SpecFlow allow abstracting system behavior, someone still needs to write the code. It became a two-pronged approach – write crafty statements in Gherkin, then code them – unnecessary costs and overhead for participating teams. The non-technical team members, for whom BDD was meant to simplify things, still found BDD an activity with a hurdle called “coding Gherkin behavior statements”.

Amidst the poor performance of BDD tools in the market, SmartBear gave up Cucumber, and Tricentis shut down the entire SpecFlow project.

Related Reads:

• SpecFlow is Dead
• Cucumber is Dead, and AI is Replacing It
• Cucumber is Dead (Reddit)
• Is SpecFlow really abandoned or is it just a catch title? (Reddit)
• “Cucumber is Dying”, What did we learn?
• SpecFlow is Dying

What’s Ahead?

So does this mean that BDD is dead too? – No.

And what about SpecFlow and Cucumber?

They’re around, too, but you might want to keep away from those tools. Especially considering that even big names don’t find these tools relevant anymore, despite investing heavily in them.

Remember, BDD is more than test automation. With changing times, BDD is evolving into something better. At the heart of it, the principle remains the same – Collectively use a common, easy language to define system behavior before beginning work.

Now, it is free from the shackles of Gherkin.

With modern technology, namely artificial intelligence (AI), there’s a lot to look forward to. You can still apply BDD without using SpecFlow or Cucumber.

Let’s take an example of a powerful AI-based test automation tool – testRigor.

You can note different expected behaviors within testRigor and use them as test cases later. One of the things that sets testRigor apart from other AI-based tools is that it lets you implement exactly what you wrote. The two-pronged approach of BDD now involves just one single step: write test steps in plain English. testRigor runs your English statements and saves you the overhead of writing code at any part of the test automation process.

Wondering how that’s possible?

Here’s how.

Supposing the product owner has brought a new requirement to test notifications of a mobile app to the team, and the 3 Amigos are having a discussion about it.

One of them opens testRigor and creates a new empty test suite with the name of the new feature whose behavior is about to be discussed.

Inside the test suite, you can create empty test cases in plain English language. This means that you can create test cases with the simple English language specifications even before test steps, UI or APIs are created to support the application functionality.

When you go to save the behavior/test case, choose the “create empty rule” option.

This will let you save the test case as is for the discussion phase. Later, the steps can be added to the test case, or you can edit the empty rules with more test steps in plain English, or you can use a host of AI features that testRigor offers to let the tool generate test steps for you based on a descriptive prompt.

Just like that you can automate everything in testRigor without writing a line of code. No Gherkin either. Just plain old English, even to refer to UI elements.

testRigor supports testing different types of applications across various platforms and browsers. You can automate many scenarios, right from filling out forms to logging in based on OTPs, all with plain English steps. This removes the need to “define” the English test steps like you had to do in Cucumber or SpecFlow. Read: How to Test Form Filling Using AI.

Since testRigor is cloud-based and has its own libraries, you need not fear a complex setup. testRigor also supports migrating your existing manual test cases from test management tools such as TestRail, PractiTest, Zephyr, etc. You can refine or even rewrite your manual test cases with reusable steps (subroutines).

Here’s a comprehensive list of testRigor’s features.

Conclusion

All good things must eventually come to an end… maybe not all good things.

Though Cucumber and SpecFlow have reached the end of their ropes – one now back in the hands of the community and the other being redone – it’s not the end of BDD. This is so because now, BDD can finally be visualized and implemented without the bindings of frameworks, like having to fit behavior descriptions into Gherkin. Instead, with AI’s backing, you can discuss and agree upon expected system behaviors, write them in a truly common language, and even automate them without extra complications.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT