Playwright is Dead

Anushree Chatterjee

Let’s talk about Playwright, a test automation tool that is gaining popularity. While it’s a relatively new player in the testing playground, introduced in 2020 by Microsoft, it’s been carving out a name for itself until recently. With many powerful test automation tools emerging, Playwright might be at a disadvantage, especially when it comes to AI-based testing.

Key Takeaways

• Playwright brought a unified API for testing across Chromium, Firefox, and WebKit, making cross-browser testing easier. It offered reliable automation features like auto-waiting, isolated browser contexts, and fast execution across platforms.
• Modern AI testing tools can now auto-generate tests from plain English, analyze failures intelligently, and adapt to changes without manual edits. Playwright, by comparison, relies on hardcoded logic and lacks native AI-driven capabilities.
• Creating tests in Playwright requires programming knowledge, limiting participation to engineers.This creates disconnects between business users, QA, and devs.
• UI changes often break Playwright tests due to their reliance on static selectors like XPaths and CSS. AI-based tools like testRigor can “self-heal” tests by identifying elements based on context and text — just like a human.
• Playwright doesn’t support native mobile or desktop testing and struggles with AI/LLM feature validation. And teams are moving toward tools that democratize testing through no-code and intelligent automation.

Let’s understand why it is so.

Playwright’s Vision

Before we jump to the problem, try to understand this tool’s offerings. Playwright aims to provide a reliable, fast, and comprehensive browser automation tool that addresses the challenges of modern web application testing. It aims to empower developers and QA engineers to build high-quality web applications by enabling robust and efficient end-to-end testing across various browsers and platforms. It offers many good features like:

• Cross-Browser Testing: Playwright’s primary goal is to provide a single API that works consistently across the major browser engines (Chromium, Firefox, and WebKit/Safari). This eliminates the need to write and maintain separate test suites for different browsers, reducing complexity and increasing efficiency.
• Platform Independence: It supports testing on Windows, Linux, and macOS, ensuring that applications function seamlessly across different operating environments.
• Isolated Environments: Each test runs in a completely isolated browser context, with its own cookies, cache, and local storage.
• Auto-Waiting: This is a cornerstone of Playwright’s vision. It automatically waits for elements to be visible, enabled, and actionable before performing operations, eliminating the need for manual, often arbitrary, waits in tests.
• Multiple Language Support: With support for TypeScript, JavaScript, Python, .NET, and Java, Playwright caters to a wide range of development teams and their preferred languages.
• Fast Execution: Designed for speed, Playwright allows for parallel test execution and efficient browser interactions, leading to faster feedback cycles.

Playwright definitely performs better than its contemporaries, like Selenium, Cypress, Cucumber, SpecFlow, and Robot.

So, where’s the problem, you may wonder…

The Problem with Playwright

With AI in the picture, the old way of coding test cases and debugging flaky runs is out the window. AI can do a lot today, create test cases in natural language, ensure stable test runs, self-heal test cases to cut down test maintenance, give intelligent analysis of test runs, and a lot more. It’s the perfect intelligent assistant for testers. When trying to measure up to such options, Playwright falls short because:

True End-to-End Testing is Still a Challenge

If you look at end-to-end testing, it inherently means going beyond testing mere clicks and form fill-ups. You need to test complex user workflows like email testing, login using TOTPs, work with files, check the UI of the application, and so much more. With a powerful AI tool (like testRigor), you can do this easily, with a single English statement. But with tools like Playwright, you need to write lines of code.

Having to Code Excludes Helpful Perspectives

A big challenge with end-to-end testing is that those who have the knowledge about how users use the application are the ones who can’t code. They have to rely on developers or test automation engineers to come up with test cases. Then there are the back-and-forths to get the right test scenarios, as those who code the test cases usually don’t have the complete picture. This is also a problem in dynamic environments where time is of the essence. With AI simplifying test creation, the prerequisite of being a programmer is no longer necessary. This naturally makes teams gravitate towards solutions that allow everyone to participate in the process without such hurdles.

const { test, expect } = require('@playwright/test');

test('Amazon search for headphones on PC page', async ({ page }) => {

  await page.goto('https://www.amazon.com/');
  await page.click('.nav-logo-link');

  // Check that page contains "Amazon Devices" below "Get to Know Us"
  const getToKnowUsText = await page.$eval('a[href="/amazon-devices/b/?ie=UTF8&node=2102313011"]', (el) => el.textContent);
  expect(getToKnowUsText).toContain('Amazon Devices');
  await page.click('a[href="/amazon-devices/b/?ie=UTF8&node=2102313011"]');

  // Check that page contains "Echo & Alexa"
  const echoAlexaText = await page.$eval('a[href="/Echo-Devices/b/?ie=UTF8&node=9818047011"]', (el) => el.textContent);
  expect(echoAlexaText).toContain('Echo & Alexa');
  await page.click('a[href="/Echo-Devices/b/?ie=UTF8&node=9818047011"]');

  // Check that page contains "Alexa App" below "Alexa Resources"
  const alexaResourcesText = await page.$eval('.s-navigation-indent-1 a[href="/gp/help/customer/display.html/?nodeId=GJDWQ6WJ6VYJX9XN"]', (el) => el.textContent);
  expect(alexaResourcesText).toContain('Alexa Resources');

  const alexaAppText = await page.$eval('.s-navigation-indent-2 a[href="/gp/help/customer/display.html/?nodeId=GDKGDTXNTH7A5BQA"]', (el) => el.textContent);
  expect(alexaAppText).toContain('Alexa App');

  await page.click('.s-navigation-indent-2 a[href="/gp/help/customer/display.html/?nodeId=GDKGDTXNTH7A5BQA"]');

  // Check that page contains "Search Amazon"
  const searchAmazonText = await page.$eval('label[for="GLUXSearchInput"]', (el) => el.textContent);
  expect(searchAmazonText).toContain('Search Amazon');

  await page.fill('#GLUXSearchInput', 'headphones');
  await page.press('#GLUXSearchInput', 'Enter');
  await page.waitForNavigation();
  await page.click('a[href*="headphones"]');
  await page.evaluate(() => window.scrollBy(0, window.innerHeight / 2));

  // Grab value from "Sony ZX Series Wired On-Ear Headphones, Black MDR-ZX110" and save it as "headphone1"
  const headphone1Text = await page.$eval('a[href*="MDR-ZX110"] span', (el) => el.textContent);
  const headphone1 = headphone1Text.trim();

  // Grab value from "Beats Solo3 Wireless On-Ear Headphones"
  const headphone2 = await page.innerText('text=Beats Solo3 Wireless On-Ear Headphones - Apple W1 Headphone Chip, Class 1 Bluetooth, 40 Hours of Listening Time, Built-in Microphone - Black (Latest Model)');
  await page.evaluate(() => window.scrollBy(0, -window.innerHeight/2));
  await page.fill('input[type="search"]', 'pc');
  await Promise.all([
    page.waitForNavigation(),
    page.click('text=pc')
  ]);

  // Check that page does not contain "headphone1"
  const hasHeadphone1 = await page.isVisible(`text=${headphone1}`);
  expect(hasHeadphone1).toBeFalsy();

  // Check that page does not contain "headphone2"
  const hasHeadphone2 = await page.isVisible(`text=${headphone2}`);
  expect(hasHeadphone2).toBeFalsy();
});

Don’t you think this is something best handled by an experienced engineer? And if that’s the case, then it totally defeats to intention of promoting collaboration and participation in end-to-end testing. Not to mention, this script also requires frequent maintenance to ensure nothing breaks in this script if the application UI or element attributes change a bit.

click "amazon"
check that page contains "Amazon Devices" below "Get to Know Us"
click "Amazon Devices"
check that page contains "Echo & Alexa"
click "Echo & Alexa"
check that page contains "Alexa App" below "Alexa Resources"
click "Alexa App"
check that page contains "Search Amazon"
enter "headphones" in "Search Amazon"
click "headphones"
scroll down by 1/2 of the screen
grab value from "Sony ZX Series Wired On-Ear Headphones, Black MDR-ZX110" and save it as "headphone_1"
grab value from "Beats Solo3 Wireless On-Ear Headphones - Apple W1 Headphone Chip, Class 1 Bluetooth, 40 Hours of Listening Time, Built-in Microphone - Black (Latest Model)" and save it as "headphone_2"
scroll up
enter "pc" into "headphones"
click on "pc"
Check that page does not contain "headphone_1"
Check that page does not contain "headphone_2"

If you want to test form filling, you need not mention all the fields one by one, use testRigor’s AI to fill in the form automatically. Read: How to Test Form Filling Using AI. That’s how straightforward it is!

Here’s another example of how easy end-to-end testing can be with an AI-based tool that doesn’t rely on coding – How to do End-to-end Testing with testRigor.

Test Maintenance is Still a Challenge

Playwright relies on explicit locators (CSS selectors, XPaths, text, roles, etc.). If the UI changes (e.g., an ID or class name is updated), those locators break, and the test fails, which requires manual updates. On the contrary, you can use AI to identify elements based on their visual appearance, text, and context, similar to how a human would. This means if a button’s ID changes but its text and location remain the same, an intelligent tool like testRigor still finds and interacts with it. This way, AI can “self-heal” tests and reduce your maintenance woes.

Read more about self-healing in testing over here – AI-Based Self-Healing for Test Automation.

Native Desktop and Mobile Testing

Along with web-based testing, you might need to check different types of applications, like those that are native to desktop or mobile. With Playwright, this is a problem. In such cases, having a versatile tool that can work for web, native, and hybrid apps is helpful. You can even test mainframe applications using testRigor’s generative AI. Here is how to do Mainframe Testing using AI.

Testing AI Features like LLMs

Software testing isn’t the only industry that uses AI. Every industry is making use of AI, as LLMs or AI agents. Hence, you might find yourself having to test such AI features. Due to AI’s dynamic nature, traditional test automation (with Playwright) may be a painful journey. Instead, you’d do much better to employ AI-based testing tools to deal with the dynamic nature of AI features. Read more about testing AI features over here – AI Features Testing: A Comprehensive Guide to Automation.

Visual Testing

Modern-day applications use many dynamic elements. While Playwright does offer screenshot-based comparisons, you might need more intelligence in visual testing. AI-powered tools tend to offer vision AI to test like a human, and can identify images like logos and perform assertions based on this.

Read more about vision AI over here – Vision AI and how testRigor uses it.

Use testRigor to test the untestable scenarios such as graphs, images, mainframes, user intentions, true/false statements, Flutter apps, and many more.

Read: How to use AI to test AI.

Conclusion

While Playwright is a powerful, flexible, and open-source tool for coded automation, AI-based tools like testRigor represent a shift towards a more intelligent, accessible, and lower-maintenance paradigm for end-to-end testing. This especially appeals to organizations looking to scale automation without scaling the engineering effort proportionally.

Even though you might hear and see accounts of people integrating Playwright with other third-party tools to try and create some of that AI-based testing experience, it is always going to be an overhead in a world where you already have AI-based tools that offer it all in a single package.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT