How to Automate Exploratory Testing with AI in testRigor

As part of extensive testing procedures, you often need to perform exploratory testing to identify issues not explicitly asked to be searched for. During exploratory testing, you must engage actively with the application and explore its functionality for issues. Those could include broken layouts, misaligned labels, typos, etc.

This method emphasizes real-time learning, test design, and execution, allowing testers to make decisions and create tests based on their discoveries and insights during test execution. Exploratory testing complements more structured testing methodologies by identifying issues that may not be covered by standard test cases, providing a deeper understanding of the application’s practical behavior in real-world scenarios.

When Should You Use Exploratory Testing?

During exploratory testing, the tester’s flexibility, creativity, expertise, and intuition are crucial for effective testing.

Early Stages of Development

• When requirements are unclear or incomplete: Exploratory testing allows testers to explore the software and identify functionality gaps or misunderstandings in the requirements before they are fully solidified.
• During rapid prototyping: When detailed specifications or documentation are unavailable, it provides immediate feedback to developers on new features or user interfaces.

Example: A startup is developing a new mobile app incorporating machine learning to suggest personalized fitness routines. Since the app is in its initial prototype phase, the requirements continually evolve based on user feedback from early users. Exploratory testing allows the testing team to interact with new features as they are developed. It provides immediate feedback to the developers and helps refine the user experience and functionality with each iteration.

Read: Shift Left Testing – Software Testing Done Right.

Complex or Innovative Products

• When dealing with new or highly innovative products: Exploratory testing helps testers understand how new technologies behave and interact with users in real-world scenarios. This is especially true when predefined test cases may not yet exist.
• In systems with complex user workflows: Exploratory testing can uncover issues that scripted testing might miss. Particularly those arising from complex user behaviors or interactions between complex system components.

Example: A company is developing a financial analytics platform that integrates data from multiple sources to provide real-time investment insights. Due to the intricacy of data interactions and the innovative nature of the analytics being used, testers employ exploratory testing to simulate advanced user scenarios. This ensures all components interact harmoniously and accurately under varied conditions.

testRigor supports testing complex financial platforms such as MyBanco, Apache Fineract, FinCell, and many more.

Agile and Iterative Development Environments

• In agile development cycles: Where rapid iteration and continuous feedback are part of the process, exploratory testing can be integrated into each sprint to assess product increments thoroughly.
• Post-release updates: In agile environments, where updates and feature additions are frequent, exploratory testing helps ensure that new integrations work seamlessly with existing functionalities.

Example: A software development team is working on an agile project to deliver a new online shopping platform. Each sprint introduces new features like payment integration, product recommendations, and customer reviews. Exploratory testing is conducted at the end of each sprint to ensure that the new features work well with the existing system and to identify any unexpected behaviors or bugs.

Read: Why Companies Switch to testRigor for E-Commerce Testing?

Usability Enhancement

• When enhancing usability and user experience: Exploratory testing allows testers to simulate real user behavior. It provides insights into the user experience that may not be captured through scripted testing.

Example: A gaming company is about to release a new adventure game. Before launch, testers conduct extensive exploratory testing focusing on user interface elements and gameplay. It helps to ensure a seamless and engaging user experience. This helps identify non-obvious bugs affecting the game’s flow under rare conditions or unique player choices.

Before Major Releases

• As a final check before launch: Exploratory testing can serve as a last-minute sanity check to catch any critical issues that might have been overlooked during formal testing phases.
• To verify fixes and regression issues: After bugs are fixed, exploratory testing can ensure that the fixes work as expected and do not introduce new issues.

Example: A software company is preparing for the major release of its updated document management system, which includes enhanced security features and integration with third-party cloud storage services. A few weeks before the launch, exploratory testing is performed extensively to ensure all elements work correctly. It discovers any issues arising from the system’s interaction with external services.

Limited Documentation or Dynamic Modules

• In projects with minimal documentation: Where documentation is lacking or outdated, exploratory testing helps testers learn about the application on the fly and test it effectively.
• Highly dynamic areas of the application: For parts of the application that are frequently changed or updated, exploratory testing can adapt quickly to new changes.

Example: A team is developing a custom CRM system for a client. The project scope and requirements have been evolving rapidly due to changing needs and additions requested by the client. With limited documentation available, testers use exploratory testing to understand the functionalities of newly added features. They ensure the features are integrated smoothly with the existing system without introducing new issues.

Features of Exploratory Testing

These are the common characteristics of exploratory testing:

• Flexibility: You can adjust your testing strategies and approaches based on the application’s behavior and any issues encountered.
• Learning-oriented: You can learn about the software’s behavior, capabilities, and limitations through active exploratory and testing.
• Adaptability: This method is beneficial when the software’s requirements are not fully defined or when dealing with complex, uncertain, or rapidly changing environments.
• Heuristic: You can use heuristics or rules of thumb to shape the testing process, which helps identify new tests or areas needing attention.
• Skill-dependent: The effectiveness of exploratory testing relies heavily on the tester’s experience, intuition, and knowledge of similar applications or systems.

Steps to Perform Exploratory Testing

Here are the traditional steps to perform exploratory testing. However, all these steps can be minimized using intelligent and AI-powered tools such as testRigor. You can easily automate these steps using testRigor’s built-in smart features, which we will discuss in the last section.

Understand the Application Thoroughly

• Get familiar with the product: Explore the application to understand its purpose, functionality, and user interface. This initial exploration helps identify potential areas for deeper testing.
• Review available documentation: Although exploratory testing doesn’t rely heavily on documentation, any available materials (like requirements, user manuals, or previous bug reports) can provide valuable insights. Read: Best Practices for Creating an Issue Ticket.

Define Testing Goals

• Set objectives: Even though exploratory testing is exploratory, having clear goals can help guide the testing process. Determine what you want to test or explore, such as new features, security aspects, user experience, or performance.
• Identify focus areas: Based on your initial understanding, decide which parts of the application are most critical or likely to have issues.

Plan Your Test Session

• Time-box your testing: Plan how much time you will spend on a testing session. This keeps your testing efforts focused and efficient.
• Prepare tools and resources: Ensure you have all necessary tools, such as test data, logging tools, and access rights, at hand. Read about Effective Test Data Management Tools.

Explore and Test

• Interact with the application: Use the application as end users would, trying out various functions and features. Pay attention to unusual behaviors or unexpected outputs.
• Take notes and screenshots: Document your findings and thoughts as you go along. This can be valuable for reporting bugs and for future test cycles.
• Vary your testing approaches: Use different testing techniques like boundary value analysis, error guessing, or different data inputs to see how the system behaves.
• Collaborate with others: Discuss your findings with other testers or developers to gain different perspectives and ideas.

Analyze Findings

• Review your notes: Look for patterns or recurring issues in your findings. This can indicate deeper problems in the application.
• Prioritize issues found: Determine which issues are most critical and should be addressed first based on their impact on the application. Read: Mastering Defect Management: Understanding Severity vs. Priority.

Report and Communicate

• Document your findings: Prepare a report summarizing the tests, the issues found, and any recommendations for further testing.
• Share insights with the team: Provide feedback to developers and other stakeholders. Effective communication can help in addressing the issues more efficiently.

Reflect and Learn

• Evaluate your testing process: Consider what worked well and what could be improved for future testing sessions. Here is a Test Automation Playbook.
• Update your test strategies: Incorporate the lessons learned into your ongoing testing practices. Read: Understanding Test Plan vs Test Strategy.

Techniques of Exploratory Testing

Below are the common techniques used for exploratory or exploratory testing:

Session-Based Testing

This method structures exploratory testing through defined sessions. Each session is time-boxed (typically between 60 to 120 minutes) and focused on specific areas or tasks. Testers are assigned a mission and goals for each session, and results are rigorously documented and reviewed. This helps maintain focus and accountability while allowing for the flexibility inherent in exploratory testing.

Role of AI

• Intelligent test automation tools such as testRigor can handle routine checks, regression testing, predefined test cases, and even exploratory testing completely.
• AI-driven analytics can evaluate test logs and session results to identify patterns or anomalies that human testers might overlook.

Pair Testing

In pair testing, two testers work together at one workstation to test the application. One operates the software while the other takes notes and thinks of new ideas to test. This collaboration can spark more ideas, uncover more defects, and lead to a deeper understanding of the application. It also helps in cross-skilling and knowledge sharing.

Role of AI

• However, this approach requires both testers to be proficient with the application. AI-powered collaboration and cloud-based automation tools such as testRigor can facilitate automated pair testing, especially in remote settings, by providing real-time suggestions, highlighting potential risk areas, and maintaining a synchronized view of testing activities. Read: Test Log Tutorial to understand how simple it is with testRigor.

Checklist-Based Testing

While exploratory testing is often seen as unscripted, using checklists can provide a balance. Checklists ensure certain key areas, functionalities, or user journeys are explored. They don’t prescribe how to test but ensure that critical aspects are not overlooked during the session.

Role of AI

• Automation can ensure that checklists are consistently followed and completed. It can also dynamically update checklists as new features are added or as new common bugs are identified.
• AI can generate checklists based on historical defect data and common failure points specific to the application or similar applications. Use testRigor’s powerful test logging and reporting to track these failure points easily.

Error Guessing

This technique relies on the tester’s experience and intuition to guess the most probable areas where bugs might occur. Testers use their knowledge of common errors in similar applications or systems to guide their testing. This can be very effective, especially in complex systems where certain types of bugs tend to recur.

Role of AI

• Machine learning models can be trained on historical test data to identify potential error-prone areas more effectively. These models can predict where errors are likely to occur based on code complexity, change frequency, and past defect trends.

Tour Testing

In tour testing, testers explore the software as if they are on a guided tour. Each “tour” focuses on a particular aspect of the application, such as:

• The User Tour: Focus on the application from an end-user’s perspective.
• The Tester Tour: Try to break the system by entering unexpected inputs, trying to overload the system, etc.
• The Architect Tour: Focus on the system’s structure and underlying frameworks.

Role of AI

• Automated scripts can simulate parts of the tours, particularly for repetitive or straightforward functionalities.
• AI can propose new tours based on changes in the application or emerging user behavior patterns. testRigor is intelligent enough to automatically cover frequently used functionality up to 1,000 test cases. If there are specific edge cases you want to discover beyond this, you can quickly create a test by using our browser plugin or modifying existing tests.

Scenario Testing

Create realistic user scenarios that reflect complex transactions or use cases involving multiple system components. Testers enact these scenarios while noting any usability issues or defects that emerge. This helps ensure the system meets real-world use and expectations.

Role of AI

• Automated scenario testing tools can execute complex user scenarios, providing detailed logs and performance metrics. AI can enhance this by dynamically modifying scenarios in real time to explore different paths based on initial outcomes.

  This is a sample testRigor test case. It is in plain English and requires no complex CSS/XPath locators for element identification.

  click "Cart"
  check that page contains "Cart Is Empty"
  enter "TV" into "Search"
  enter enter
  click "TV"
  click "Add To Cart"
  check that page contains "Item Added To Cart"
  check that page contains "Cart (1)"

testRigor efficiently manages the 2FA scenarios, QR Code, email, phone calls, SMS, file upload, database testing, and Captcha resolution, and many more complex scenarios through its simple English commands.

Charter-Based Testing

This approach involves creating a test charter that outlines a test session’s scope, objectives, and time limit. The charter helps guide the tester’s efforts and can be very specific (like testing a particular feature) or broad (like assessing overall usability).

Role of AI

• AI can help craft test charters by analyzing project documentation and codebase changes to recommend focus areas. Automated tools can track the execution against these charters and report on compliance and coverage. You can use generative AI-powered tools such as testRigor to generate test cases in plain English based on the provided description. See here how testRigor supports SDD (Specification-Driven Development).

Automate Exploratory Testing with testRigor’s AI

Exploratory testing is traditionally considered a manual job, where testers use their skills and intuition to perform effective application testing. However, thanks to recent advancements in AI, it is now possible to smartly automate it.

With testRigor, exploratory testing is quick and super easy.

Let us see how.

Step 1: Log in to your testRigor account if you are already registered. Otherwise, use this link to sign up.

Step 2: First, we will create a new test suite and a sample test case. Enter the below information while creating a new test suite:

• Test Suite Name: Provide a relevant and self-explanatory name.
• Type of testing: Select from the following options: Desktop Web Testing, Mobile Web Testing, Native and Hybrid Mobile, based on your test requirements.
• URL to run test on: Provide the application URL that you want to test.
• Testing credentials for your web/mobile app to test functionality which requires user to login: You can provide the user login credentials of Application Under Test (AUT) here and need not write them separately in the test steps then. The login functionality will be taken care of automatically using the keyword login in test steps.
• OS and Browser: Choose the OS Browser combination on which you want to run the test cases.
• Number of test cases to generate using AI: If you wish, you can choose to generate test cases based on the App Description text, which works on generative AI. For now, we are not using this option.

Click Create Test Suite.

Step 2: Click Add Custom Test Case to create a new test case in this test suite.

Step 3: Add plain English test steps for the test case, as shown below. You can also import/ copy your manual test cases from all test management tools such as TestRail, Qase, PractiTest, etc.

Step 4: The test execution details, logs, reports, video, error text, and test execution video are available on the Test Cases screen.

Read this step by step guide to know how to perform End-to-End testing using testRigor.

Step 5: To enable Exploratory Testing go to Settings -> Error reporting.

Select a value for ‘Automatically detect UI issues using Vision AI on every page:’ based on your requirement. Next, select how this detection will map to reported issues by selecting a value in ‘Cap Vision AI-detected UI issues:’.

Click Save to use these settings in the next test runs.

Step 6: Navigate to the Test Cases tab on the left-hand side panel and click Re-test. Enjoy!

After the test is executed, the test results will show the Old and New UI screenshots with error information (if any) and classifications (Major, Minor, etc., as per your selection in Step 5).

testRigor enables you to test web, mobile (hybrid, native), API, and desktop apps with minimum effort, time, and maintenance.