ChatGPT for Test Automation
It is estimated that training the OpenAI ChatGPT model happened in a mere 34 days. Interestingly, the company’s executives predict reaching a revenue of $5 billion by the end of 2024. Every industry and organization uses ChatGPT for some purpose or another. Understanding whether we can use ChatGPT in software testing activities would be interesting.
Let us find out in this article.
What is ChatGPT, and how does it work?
Here is a response from the ChatGPT, what it assumes itself to be:
ChatGPT is an AI language model developed by OpenAI, designed to generate text that mimics human-like responses in a conversational format. It’s built on the Generative Pre-trained Transformer (GPT) architecture, enabling it to understand and produce text across a wide range of topics and styles. ChatGPT can engage in dialogues, answer questions, provide explanations, and even create content such as stories, essays, or code, based on the input it receives.
The model is pre-trained on a diverse dataset compiled from the internet, which helps it learn language patterns, context, and information across numerous subjects. However, its responses are generated based on patterns in the data it has been trained on; it doesn’t possess understanding or awareness but rather predicts text that is statistically likely to follow the given prompt.
Hence, ChatGPT is a popular machine-learning model that is trained to chat with people and produce correct-looking results. As you can see above, it produced an impressive explanation and is learning fast.
Does it mean that ChatGPT can be trusted?
So, how does ChatGPT work? It is a neural network trained on massive data to answer questions and hold context to a certain degree. In essence, it is a set of functions extrapolating based on “training points“.
This is where ChatGPT’s pitfalls and limitations arise: since it is not nearly as powerful as a human brain, it does not have a high enough degree of complexity to be able to validate and understand either the question or the answer, and therefore, often produces non-sensical results that look legitimate. It becomes hazardous when the most part of the answer looks correct, but the issues are somewhere in the middle of the text. In fact, Stack Overflow banned ChatGPT for “constantly giving wrong answers”.
Using ChatGPT to create test scripts
Similarly to the description above, ChatGTP can produce some results based on its training that can be very helpful in jump-starting your test. Here is an example of a sign-up test in Selenium using Java:
Below is an example of a sign-up test using Selenium with Java. This test assumes you’re testing a web application with a sign-up form that includes fields for a username, email, password, and a sign-up button. Please adjust the element IDs (‘username’, ’email’, ‘password’, ‘signUpButton’) and URL (‘http://example.com/signup’) to match your actual application.
import org.openqa.selenium.By; import org.openqa.selenium.WebDriver; import org.openqa.selenium.WebElement; import org.openqa.selenium.chrome.ChromeDriver; import org.openqa.selenium.support.ui.WebDriverWait; import org.openqa.selenium.support.ui.ExpectedConditions; import org.junit.After; import org.junit.Before; import org.junit.Test; import static org.junit.Assert.*; public class SignUpTest { private WebDriver driver; @BeforeMethod public void setUp() { // Set path to your chromedriver.exe System.setProperty("webdriver.chrome.driver", "/path/to/chromedriver"); // Initialize the driver driver = new ChromeDriver(); // Maximize the browser window driver.manage().window().maximize(); } @Test public void testSignUp() { driver.get("https://www.example.com/signup"); // Fill out the form driver.findElement(By.id("username")).sendKeys("testUser"); driver.findElement(By.id("email")).sendKeys("[email protected]"); driver.findElement(By.id("password")).sendKeys("securePassword123"); // Click the sign-up button driver.findElement(By.id("signUpButton")).click(); // Wait for navigation and assert a successful sign-up // This could be a redirection to a dashboard, login page, or displaying a success message. // Adjust the condition to fit your application's sign-up flow new WebDriverWait(driver, 10).until(ExpectedConditions.urlToBe("http://example.com/dashboard")); // For example, check if the success message is displayed WebElement successMessage = driver.findElement(By.id("successMessage")); assertTrue(successMessage.isDisplayed()); } @After public void tearDown() { // Close the browser driver.quit(); } }
This script does the following:
- Initializes a new ChromeDriver instance to control Chrome.
- Navigates to the sign-up page of your application.
- Finds the username, email, and password input fields and the sign-up button by their IDs, and interacts with them by entering data and clicking the button.
- Waits for a specific condition to be met after the sign-up button is clicked, such as being redirected to a success page or a success message being displayed on the screen. This is where you should customize the script based on how your application confirms a successful sign-up.
- Asserts that the sign-up was successful, for instance, by checking the presence and visibility of a success message. This part of the script must also be tailored to fit the specifics of your application’s UI and flow.
- Finally, it closes the browser window.
In this example, we use the ChromeDriver class from the Selenium library to launch the Chrome browser and interact with the website. The @Before and @After annotations are used to set up and tear down the WebDriver instance before and after each test.
In the testSignUp method, we navigate to the sign-up page, locate the email, password, and sign-up button elements on the page using the findElement method, enter values into the email and password fields, and click the sign-up button.
Finally, we locate the success message element and use the assertTrue method to verify that the text of the success message is “Sign up successful!“.
As you can see, it is helpful, although it has several issues:
- It relies on element ids in the code for element identification; hence, any element attribute change may result in an unstable script and code maintenance.
- It doesn’t generate a unique username/email and uses the hard-coded value every time. Hence, it will fail in the subsequent test runs in a typical test environment.
- It is incomplete, as there is no email confirmation validation or to check that the newly created credentials can be used in the application for login.
What is testRigor?
testRigor is an AI agent that executes plain English commands in the test case precisely like a human would. testRigor is powered by generative AI and is an intelligent end-to-end testing tool. It allows you to perform web, mobile (native, hybrid), desktop, and API testing singlehandedly in plain English.
login create a new lead select "Mr" from "Salutation" generate unique name, then enter into "First Name" and save as "firstName" generate unique name, then enter into "Last Name" and save as "lastName" generate from template "###-###-####", then enter into "Phone" and save as "generatedPhone" enter "testRigor" into "Company" click "Save" check that page contains "created" check that page contains stored value "firstName"
As you can see, the test steps are simple. This empowers the testers or anyone on the team to use testRigor as a test automation tool without dedicating time to programming and debugging exceptions. Just register and start creating the tests in plain English. You can also import your manual test cases and, with some adjustments, use them as testRigor’s test cases. Read here about testRigor as an automation testing tool for manual testers.
Here are a few exciting capabilities of testRigor:
- Email, Phone Call, and SMS Testing: You can use simple English commands to test the email, phone calls, and SMS. These commands help validate 2FA scenarios, with OTPs and authentication codes being sent via email, phone calls, or text.
- File Upload/ Download Testing: Execute the test steps involving file download or file upload without the requirement of any third-party software. You can also validate the contents of the files using testRigor’s simple English commands.
- Database Testing: Execute database queries and validate the results fetched.
- Reusable Rules (Subroutines): Create functions for the test steps you use repeatedly. You can use the Reusable Rules to create such functions and call them in test cases by simply writing their names. See the example of Reusable Rules.
- Global Variables and Data Sets: You can import data from external files or create your own global variables and data sets in testRigor for data-driven testing.
- 2FA, QR Code, and Captcha Resolution: testRigor efficiently manages the 2FA, QR Code, and Captcha resolution through simple English commands.
testRigor vs. ChatGPT
click "Sign up"
generate unique name, then enter into "Username" and save as "username"
generate unique email, then enter into "Email" and save as "generatedEmail"
generate unique name, then enter into "Name" and save as "generatedName"
enter "PasswordSuperSecure!" into "Password"
save value "PasswordSuperSecure!" as "password"
click "Submit"
check that page contains "Sign up successful!"
check that email to stored value "generatedEmail" was delivered
click "Confirm email"
check that page contains "Email was confirmed"
check that page contains expression "Hello, ${generatedName}"
click "Logout"
login
check that page contains stored value "generatedName"
As you can see, using testRigor, you can outline the steps and follow them the way you described. Its goal is to allow you to create high-quality test cases quickly and precisely execute test steps without writing any code or relying on implementation details.
Additionally, if attribute changes happen, such as a ‘Submit’ button being changed from <button> tag to <a> tag, testRigor, as an AI agent, would understand the button’s role within the application context. Since the elements are identified using the text visible on the screen, even if the attribute changes, the element remains identifiable with the exact text.
Let us hear from the ChatGPT itself what it thinks about testRigor vs. ChatGPT for software testing:
- Automated Test Case Generation: testRigor allows users to describe tests in plain English, which it then converts into automated test scripts. This feature lowers the barrier to entry for writing automated tests and makes the process faster and more accessible to non-technical stakeholders.
- UI and API Testing: It provides capabilities for both user interface (UI) testing and application programming interface (API) testing. This allows teams to test the front-end and back-end components of their applications comprehensively.
- Cross-Browser and Mobile Testing: It supports testing across multiple browsers and mobile devices, helping teams ensure their applications work seamlessly across different environments and platforms.
- Visual Testing: testRigor includes visual testing features to detect UI changes or issues that might not be caught by traditional functional testing, such as layout issues, color mismatches, and more.
- Self-Healing Tests: The platform employs advanced algorithms to automatically adjust to changes in the application’s UI, reducing the need for manual test maintenance and increasing the resilience of test suites over time.
- CI/CD Integration: testRigor can be integrated with continuous integration and continuous delivery (CI/CD) pipelines, allowing automated tests to run as part of the software release process. This helps in identifying and fixing issues early in the development cycle.
- Real User Simulation: It can simulate real user behaviors and interactions more accurately than a general-purpose AI like ChatGPT, which is crucial for identifying usability issues and ensuring a positive user experience.
- Detailed Reports and Analytics: testRigor provides detailed reports and analytics on test executions, including screenshots, logs, and performance metrics, which are essential for troubleshooting and improving test coverage.
In summary, while ChatGPT can be a valuable tool for generating test ideas, explaining testing concepts, or even writing simple test scripts, testRigor offers a comprehensive suite of features specifically designed for automated software testing. These features make testRigor better suited for organizations looking to streamline their testing processes, improve test coverage, and enhance the quality of their software products.
You can also refer to more testRigor’s top features here.
Conclusion
ChatGPT is a good companion, and its intelligent use can propel the testing activities with great acceleration. However, you can not rely entirely on ChatGPT for script generation or testing. You can use it to suggest applicable test scenarios for an application and use them to create test cases further using intelligent test automation tools such as testRigor.
AI and test automation are helpers of humans, and it is up to us how we utilize the tools available to us currently. Nothing can replace human intelligence, but these tools can surely aid in performing mundane and repetitive tasks so that humans can focus on what we can do better.
Thank you for your request. Please select date and time for a demo.
