Why Cypress sucks for end-to-end tests in 2024

Hari Mahesh

End-to-end (E2E) Testing is a critical aspect of the software development lifecycle, ensuring that an application functions as intended from the user’s perspective across all layers of the system. The importance of E2E testing lies in its ability to provide comprehensive coverage, reduce risk, and improve overall software quality. By mimicking real user behavior, E2E tests ensure that critical business processes and user journeys perform correctly, even as the application evolves.

Hence, the automation tool we select for performing E2E testing is crucial. Cypress is one such tool available on the market. Let’s know more about Cypress and why its usage in 2024 can be not as helpful.

About Cypress

Cypress is an open-source testing framework primarily used for end-to-end testing of web applications, developed by Cypress.io, a company founded by Brian Mann in 2014. It provides a developer-friendly environment with a fast and easy setup, real-time reloads, and an interactive test runner, allowing for quick debugging. Cypress supports JavaScript and runs directly in the browser, giving it direct access to the DOM, network requests, and browser behavior, which helps ensure accurate test execution

Creating Test Scripts using Cypress

Cypress uses JavaScript as the scripting language. Cypress provides a built-in test runner and a rich set of APIs to perform various actions like visiting web pages, interacting with elements, and asserting conditions. Cypress works by element interactions and then performing the validations as per the test case. The element interactions and validations are typically performed using a series of commands that simulate user actions (such as clicking, typing, selecting, etc.) and then performing assertions to validate the application’s behavior or state.

cy.get('selector').should('assertion', 'expected value');

To write this one particular line, firstly you need to identify the element selector. Which is nothing but the element locator value. In Cypress, element locators are used to find and interact with elements on a web page. Locators can be based on various attributes such as IDs, classes, text content, or other HTML attributes. Here are the common ways to locate elements in Cypress:

• cy.get('#elementId'): Selects the element with ID ‘elementId’
• cy.get('.className'): Selects the element with class ‘className’
• cy.get('button'): Selects all <button> elements
• cy.get('input[name="username"]'): Selects the input element with the name attribute ‘username’
• cy.get('button'): Selects all <button> elements
• cy.get('input[name="username"]'): Selects the input element with the name attribute ‘username’
• cy.get('form#loginForm button.submit'): Selects the button with class ‘submit’ inside the form with ID ‘loginForm’

describe('Cypress Element Locators Example', () => {
  it('Finds elements using different locators', () => {
    cy.visit('https://example.com'); // Visit the website
    cy.get('#username').type('testuser'); // Selects the input by ID and types text
    cy.get('.login-button').click(); // Selects the button by class and clicks it
    cy.contains('Dashboard').should('be.visible'); // Finds an element with text 'Dashboard' and asserts it is visible
    cy.get('ul#menu').find('li').eq(1).click(); // Selects the second list item within the menu and clicks it
    cy.get('[data-cy="logout-button"]').should('exist'); // Asserts that the logout button with the data-cy attribute exists
  });
});

Do you think it is too wordy and confusing for a beginner automation engineer or manual tester? Finding the right locator using the old ways (id, data-testid, CSS/Xpath) can be cumbersome for modern dynamic websites.

Here is an example that shows the code complexity of the Amazon.com website. We can see 10+ levels of nested div elements. Just imagine using CSS/XPath locators to identify an element. Also, if any of the nested elements change, test maintenance will be a nightmare.

Limitations of Cypress

describe('Amazon Add to Cart and Checkout', () => {
  before(() => {

    // Visit Amazon and log in with valid credentials
    cy.visit('https://www.amazon.com');
    cy.get('#nav-link-accountList').click(); // Click on the sign-in link
    
    // Enter login credentials
    cy.get('#ap_email').type('[email protected]'); // Replace with your email
    cy.get('#continue').click(); // Click continue button
    cy.get('#ap_password').type('your-password', { log: false }); // Replace with your password
    cy.get('#signInSubmit').click(); // Click sign-in button
    
    // Ensure login is successful
    cy.url().should('include', 'your-account');
  });

  it('Adds an item to the cart and proceeds to checkout', () => {
    
    // Step 1: Search for a product
    cy.get('input#twotabsearchtextbox').type('laptop{enter}'); // Search for a 'laptop'
    
    // Step 2: Click on the first product in the search results
    cy.get('.s-main-slot .s-result-item').first().find('h2 a').click();
    
    // Step 3: Add the product to the cart
    cy.get('#add-to-cart-button').click(); // Click on the 'Add to Cart' button
    
    // Step 4: Go to the cart
    cy.get('#nav-cart').click(); // Click on the cart icon
    
    // Step 5: Proceed to checkout
    cy.get('input[name="proceedToRetailCheckout"]').click(); // Click on the 'Proceed to Checkout' button
    
    // Step 6: Select a shipping address
    cy.get('.address-book-entry').first().find('input').click(); // Select the first address
    
    // Step 7: Continue to the payment page
    cy.get('input[name="continue-top"]').click(); // Click on the 'Continue' button
    
    // Ensure the payment page is loaded
    cy.url().should('include', 'payment');
  });

  after(() => {
  
    // Log out after the test
    cy.get('#nav-link-accountList').click(); // Click on the account list
    cy.get('#nav-item-signout').click(); // Click on sign out
  });
});

If you see the above script, its a big one and it requires frequent maintenance to ensure nothing breaks in this script if application UI or element attributes change a bit. The problem with CSS locators is that they describe how to find an element in the internal HTML/XML structure of the screen as opposed to how an actual user would find it. This basically means that you are relying on details of implementation, how engineers wrote the code yesterday to describe how the test should work.Therefore, test script maintenance becomes a major headache with this tool.

Without Self-healing capabilities in the test automation tool, if any element property changes, the test case fails because of the script error. A tester needs to ensure the locators are correct before running the script. This maintenance eats up the time, effort and cost of the team working on it. Also, it adversely affects the new scripts creation due to unwanted test script maintenance.

Not the above one alone, there are many other limitations in using Cypress.

• Cypress only supports a few browsers: Cypress supports testing only in a few specific browsers: Chrome, Edge, Firefox, and Electron. This means that it does not provide support for some other popular browsers, such as Internet Explorer. For Safari, Cypress supports Webkit, that also its in experimental stage. As a result, if your application needs to be tested across a wider variety of browsers to ensure compatibility, Cypress might not be the best choice.
  For comprehensive browser testing, additional tools or frameworks may be needed to cover the unsupported browsers.
• No support for mobile applications: Cypress does not support native mobile app testing. While you can run tests in a mobile browser viewport, it is not the same as testing on actual mobile devices or emulators.
• Restrictions on testing multiple domains: Cypress cannot visit different domains in the same test due to security restrictions (same-origin policy). This means that if your test workflow involves navigating between different domains (e.g., from a marketing site to an authentication domain), Cypress might not be able to handle this scenario directly.
• No support for multiple browser instances: Cypress cannot run tests across multiple browser instances at the same time, which means it is not suitable for testing scenarios that require multiple users interacting with the application simultaneously.
  For example, if an application needs to test how different users interact in real-time, such as in a chat application or collaborative platform, Cypress won’t be able to simulate these actions in separate browser windows or tabs concurrently.
• Dependency Issues: Cypress relies on several different modules as dependencies, such as Chai for writing assertions. This means that the testing team must be careful whenever there is a version upgrade for these dependencies, as an update could cause the test scripts to become flaky or unreliable. There is also the risk that a new version of a dependency may not work well with the current version of Cypress, leading to compatibility issues. As a result, managing and keeping these dependencies up to date without breaking the tests can become an additional challenge for the testing team.
• CI/ CD Integrations: While Cypress can be set up to run tests as part of a continuous integration or continuous deployment process, it may require additional configuration to ensure compatibility with different environments. Cypress tests can be resource-intensive, consuming a significant amount of memory and CPU, which can slow down the CI/CD pipeline, especially if tests are run in parallel or on shared infrastructure.
  Also, Cypress runs in a headless browser by default in CI environments, which may lead to differences in test behavior compared to running in a full browser, potentially causing flaky tests. Additionally, troubleshooting test failures in CI/CD can be more difficult due to limited access to real-time debugging tools.
• High Resource Usage: Cypress can be resource-intensive, requiring a significant amount of memory and CPU power, especially for large test suites or complex applications. This can slow down the test execution and may require more powerful infrastructure.

How Should End-to-End Testing Work?

What is the purpose of end-to-end tests? They are meant to validate that your app’s functionality works as intended from an end-user’s perspective, following the actual user activity flow in real-world scenarios.

This means you should reference elements in the way the user experiences them, not how a developer views the application. For users, the only thing that matter is identifying the right input fields or finding the correct button to click. Therefore, it’s essential to have a simple, reliable way to interact with forms, tables, files, etc. that mimics how a user would interact with a browser or device.

Example: Form Interaction

Let us take an example of a website form to understand.

As an end user you will navigate to each field and fill in the data such as ‘City’, you are not concerned about the id/CSS/XPath etc. of ‘City’ form element. Here is the HTML of the form:

In this HTML ‘City’ element’s ‘id’ and ‘name’ are clearly mentioned. But this won’t be the case with every application.

Also, if there is a change in UI framework (as in React), you might lose all these IDs. Another case is that you migrate to a rigid back-end framework or upgrade to a newer version (ASP.NET), you might be forced to change these element names as well. Ironically, this is precisely when you need your end-to-end tests to work, i.e., right after migrating to a new framework. You need E2E tests to rely on and ensure everything functions as expected.

And therefore, your end-to-end testing tool shouldn’t depend on the internal code of your application. Instead, it should clearly mimic the perspective and actions of the end user. Take the “City” input field in the screenshot, for example. Its internal structure could change, but there will likely always be a placeholder or label that the user identifies as “City.” However, not every site has a well-structured HTML setup, like Amazon with its ‘label for’ attributes. You can’t always rely on the underlying code structure.

What’s really needed is a way to identify and describe elements from the end user’s point of view, based on what they see: such as a label or placeholder.

enter "San Francisco" into "City"

Isn’t it really simple and intuitive? Let us see another example.

Example: Table Handling

Here we are looking at a Salesforce table:

Now, from a user’s perspective, what matters is that the row containing Company as “ProperUniqueCompany” displays a specific ‘Lead Status’. Similarly, the down arrow icon in the last column of that row should be clickable. They are not worried about the implementation of these fields they just want to access and take some actions.

validate that table at row containing "ProperUniqueCompany" and column "Lead Status" contains "Open - Not Contacted"

click on the table at the row containing "ProperUniqueCompany" and the last column

This is how humans think and take action, and it should be tested in the same manner. This command will work no matter how the table is rendered – whether it’s HTML <table> (like in Salesforce example) or using <div> rendering (like in Amazon example).

testRigor: Simplifying End-to-End Testing

Today you have test automation tools powered by Artificial intelligence and Machine Learning algorithms. These tools are simple to use and therefore help you achieve greater test coverage quickly. If you are looking for an intelligent test automation tool, then the better option will be testRigor. It is packed with many advanced features like Natural Language Processing, Self Healing, AI-features testing, LLM testing, and many more.

The English commands that you have seen above are actual testRigor commands. We have seen that there are many reasons why Cypress is not suitable for E2E testing today. The tool that you use for E2E testing should be as simple as possible and help minimize the number of bugs in production.

It is possible with the available advanced AI technologies and intelligent AI agents. testRigor is a codeless automation tool powered by generative AI and ML technologies that help generate/create easy and automatic test scripts and data in plain English. With testRigor, test script maintenance requires almost zero effort. Read here how you can decrease 99.5% of your test maintenance time.

Let’s take a look at its capabilities and see how easy it is to use.

You have the following advantages with testRigor:

• No programming required: It is a codeless tool, meaning tests are written/generated using plain English commands. This opens the ability for the entire team to author tests (including manual testers) and significantly increases the speed of creating tests.
  Its record-and-playback feature can further speed up test creation. Since the recorded tests will be in the same plain English format, it becomes viable for anyone to edit and maintain when desired.
• Stable locators: There is no hassle mentioning CSS or any technical parameters for locating elements on the screen. All you need to do is mention relative positions or how you see an element on screen. You can click on a button below the title by simply writing ‘click “button” below “Title”‘.
• AI-based self healing: Using Vision AI and Auto-Healing for rules and single commands, testRigor will be able to look on the screen for the alternative way of doing what was intended as opposed to failing. This will allow you to very quickly adapt to breaking changes in your application. Read more about AI-based self-healing.
• Cross-browser and cross-platform support: The tool allows you to perform cross-browser and cross-platform tests; parallel execution feature allows you to get the test results in minutes.
• No installation needed: Being a cloud-based tool, you just need to register and get started with test automation in no time.
• Supported integrations: This can be easily integrated with most CI/CD tools, test management tools, and issue-tracking tools.
• Advanced reporting and logging: It offers good reporting capabilities and captures screenshots at every step. It provides clear error messages in English. You can also view the video recordings or error logs of the execution for clarity.
• A single tool for all testing needs: You can write test cases across platforms: web, mobile (hybrid, native), API, desktop apps, and browsers using the same tool in plain English statements.
• Test AI features: This is an era of LLMs and using testRigor you can even test LLMs such as chatbots, user sentiment (positive/negative), true or false statements, etc. Read: AI Features Testing and security testing LLMs.

Writing tests with testRigor

Writing tests with testRigor is as simple as you are describing the steps to access the application to a friend in English. The testRigor documentation has all the commands in detail, and you can even define your own preferred commands (subroutines) in plain English.

click "Sign in"
enter "[email protected]" in "email"
click "Continue"
enter "your-password" in "password"
click "Submit"
check the page contains "Your Account"
enter "laptop" in "Search Amazon"
click the first "hp laptop"
click "add to cart"
click "go to cart"
click "checkout"
click the first checkbox below "address"

Read How to do End-to-end Testing with testRigor for more details. Know more about exciting features of testRigor.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT