TestRail Overview

Anushree Chatterjee

Having the assistance of tools to perform tasks is a true luxury. This holds good in every industry, even software testing. There are a variety of tools available to simplify different aspects of software testing, like test case creation, test execution, test reporting, test maintenance, and so on.

Test management is one of these aspects of software testing that require tools to help manage, organize, and maintain test cases that form the very foundation for testing. One popular test management tool is TestRail.

What is TestRail?

TestRail is a web-based test management tool. It provides a central location for testers, developers, and other project stakeholders to collaborate on creating, organizing, and running tests.

Benefits of using TestRail

TestRail can significantly improve the quality of your software testing in the following ways:

• A central repository for organized testing: TestRail provides a central repository for managing all your test cases, suites, and runs. This eliminates the need for scattered spreadsheets or documents, improving overall organization and accessibility.
• Easier communication and collaboration: Team members like testers, developers, and project managers can collaborate effectively by sharing test cases, tracking progress, and leaving comments and discussions directly within the platform.
• Streamlined test execution and tracking: TestRail simplifies running tests, logging results, and assigning tasks to testers. This streamlines the testing process and provides clear visibility into progress.
• Efficient defect management: Integration with bug-tracking tools helps defect identification, reporting, and tracking, which facilitates faster resolution.
• Actionable insights and reporting: TestRail’s reporting features offer valuable insights into testing progress, defect trends, and test coverage. This empowers teams to make data-driven decisions and improve testing efficiency.
• Increased productivity and time-saving: Features like pre-defined test steps, version control, and scheduling can save testers significant time on repetitive tasks.
• Scalable and flexible solution: TestRail can adapt to the needs of growing teams and projects. It supports various testing methodologies (manual, automated, exploratory) and integrates with other development tools.
• Cost-effective: Several TestRail pricing options cater to different team sizes and needs, making it a cost-effective solution for many organizations.

Features of TestRail

TestRail allows you to streamline your testing process with these features:

• Manage test cases: You can create and manage test cases within TestRail.
• Version control: Track changes made to your test cases over time.
• Import/export test cases: You can easily import test cases from spreadsheets (e.g., Excel) or export them for sharing.
• Manage custom fields: Adding specific fields to capture additional test data (e.g., severity, priority) within test cases is possible.
• Organize tests: TestRail offers neat ways to organize your test cases.
  • Test suites: Group related test cases together based on functionality, features, or any other criteria.
  • Test plans: Define a set of test suites to be executed for a specific release or milestone.
  • Sections and milestones: Further organize test suites into sections and assign milestones for progress tracking.
• Track test execution: You can assign test cases to testers and track their execution progress.
• Record results: Testers can record results (pass, fail, blocked) for each test case along with comments and attachments (screenshots, logs).
• Defect reporting: There are provisions within TestRail to link test results with defect tracking systems to report and track issues.
• Schedule test runs: You can even plan and schedule test runs for specific dates and times.
• Automated testing integration: You can integrate with testing frameworks to capture automated test results within TestRail.
• Generate test reports: Creating reports on test execution progress, defect rates, and overall testing trends is possible with TestRail.
• Email notifications: Configure email notifications to alert users about test run assignments, test results, and other events.
• Custom dashboards: You can create and customize dashboards to display key metrics and reports.
• Customizable filters: You can even filter reports based on specific criteria (e.g., tester, test suite, date).
• Role-based access control: You can assign different user roles with varying access levels (e.g., tester, administrator).
• Comments and discussions: Testers can add comments and discussions within test cases for collaborative troubleshooting.
• API access: Leverage TestRail’s API for custom integrations and automation within your testing workflow.
• Cloning test cases: Clone existing test cases to create similar ones without starting from scratch.

Overview of TestRail

Here’s a walkthrough of how you can work with TestRail. Firstly, you need to log into your TestRail account. If you don’t have one, you can try their trial version before investing in the paid plans.

Create a project

Step 1: Once you log in, you will see your DASHBOARD. Over here, create a new project by clicking on ‘+ Add Project’.

Step 2: You’ll see various configurations available. To configure your project, add a project name.

Scrolling down, you can choose how you’d like TestRail to organize your test cases. Choose an option and then click on Add Project.

As soon as you do that, you’ll get a message saying that your project was created successfully and this new project will be visible to you in the list of projects.

Add test suite

Step 1: To add test suites to a project, you can do it by going to the Dashboard where your project is listed, there are a couple of options, one of them for adding test suites.

Step 2: On being redirected to the test suite page, you will see options to create a test suite for your project. Click ‘+ Add Test Suite‘.

Step 3: Provide details for the test suite and click Add Test Suite.

Under the Test Suites and Cases section within the project, you’ll see this newly created test suite.

Add test case

Step 1: Go into your project. You can do this by clicking on the project of choice on the Dashboard and then on Test Suites and Cases within the project.

Step 2: If you’ve created test suites, then select the test suite you want to add test cases to.

Step 3: Start adding test cases for the test suite. You can even import test cases using TestRail’s import capabilities.

Step 4: Add details for the test case.

Step 5: Save your test case. You can even save the existing test case and directly have TestRail open a new form for you to create another test case.

Once you create a test case, it will look like this:

Add test run

So far, you have a project that contains test suites, and each of these test suites contains test cases. Now, you need to create test runs that will help you track test executions.

Step 1: Click on Test Runs and Results and to add a test run for the test suite you want, click ‘ + Add Test Run.

Step 2: Fill in the details for the test run and choose what test cases you want to add as a part of this run. Click Add Test Run.

This is how the test runs will look like:

Add test case result

The test cases assigned as a part of a test run need to be updated. There are different statuses available based on your findings, such as Pass, Fail, Blocked, or Retest.

Step 1: Use the ‘Add Result‘ to add the test case result, or use the ‘Pass & Next‘ drop-down. You can also assign this test case to another team member through the ‘Assign To‘. There is an option to provide ‘Results & Comments‘ in the text for better understanding.

For now, we will click ‘+ Add Result‘, and we will pass the test case after adding comments and relevant files.

Once statuses are updated for the test cases, you will see the updated charts showing the progress under ‘Test Runs and Results‘.

Tracking and reporting

To further organize and share your testing operations, use Milestones and Reports tab.

Milestones help you create goals, which you can achieve through regular test runs.

TestRail offers various reports for analyzing testing outcomes. You can even mail them.

Integrating TestRail with test automation tools

Test management alone isn’t enough to improve software testing. You need to be able to integrate with other testing tools to make the process efficient. Luckily, TestRail integrates with many test automation tools, one of which is testRigor.

Integrating TestRail with testRigor

testRigor is a champion if you are looking for effortless automated testing. This AI-based tool is a cloud platform that is easy to use. It allows you to create various types of tests for different types of apps, browsers, and platforms in plain English without requiring any kind of coding.

Of the many integrations that testRigor supports, one is for TestRail. Here’s how you can do it.

Step 1: Log into your testRigor account and create a test suite where you want to import your TestRail test cases.

Step 2: Within the test suite, navigate to the Settings -> Integrations -> TestRail section. Refer to this detailed documentation on how to fill up the details in this section.

Step 3: Once you save these changes, come back to the Test Cases section. A new tab for importing TestRail test cases will be visible now.

Step 4: Click on the ‘Import test cases from TestRail’ button and choose the way you want to import these test cases.

Step 5: Once you’re ready, click on ‘Import‘. The test cases will appear in the test suite.

Here’s a detailed guide on the TestRail to testRigor test case importing. Through this integration and import you can turn your manual TestRail test cases into fully automated tests immediately with small tweaks, that too in plain English. Once you’re done with the import operation, the test cases will appear in the ‘Not started‘ state. In case testRigor cannot identify any test case line, then it will convert it into an AI-based reusable rule. This means that the statement in question is now being looked at as a series of statements that are also written in plain English and can be reused across test cases. You can edit such reusable rules to your liking, too. Can it get simpler than this?

testRigor also offers powerful test reporting, which gives you a straightforward summary of your test cases’ performance. That’s not all. You can do a lot more with this tool. Here’s a full feature list of testRigor’s capabilities.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT