 |
A Test Log is a testing artifact that chronologically records all the events and results that have occurred during the testing process, i.e., test runs. These logs are an integral deliverable of the testing cycle, which is available to the team members, management, and other stakeholders.
This article discusses the test logs in detail, their need, the information they contain, and the best practices.
What are Test Logs?
They are detailed records of the process and results of testing activities. They provide detailed insight into which tests were executed, when they were performed, what the end result of each test is, issues encountered during test runs, error messages, etc. These logs help in quality control, test management, debugging, audits, future references, etc.
Why do We Need Test Logs?
Below are the reasons why we need test logs during testing:
- Tracking the Progress: Test logs help in monitoring the progress of testing activities. It also helps in identifying the owner of each test run.
- Issue Identification: Test logs record or document the failures and help in the identification of issues/bugs. This helps developers to receive detailed information, fix bugs, and improve product quality.
- Review and Future Reference: They provide the execution details for future reference to analyze past testing efforts, results, errors, etc.
- To Meet Compliance and Audit: In the case of regulated domains and industries, these logs serve as proof for audits and compliance standards. They guarantee that the required testing was done and the results were as expected based on the regulations.
- Reporting and Analysis: The detailed information provides sufficient reporting to stakeholders to make informed decisions about the software release readiness.
What Information Test Logs Contain?
The below information is contained in the test logs:
- Test Case ID: It identifies the test case uniquely, and a test log is related to it for easy identification.
- When, Which, Who of Test Cases: Test Logs contain when the test was executed, which environment (OS, software version, etc.) was used, and who performed the test.
- Test Result of the Test Run: The outcome of each test run, i.e., Passed, Failed, Incomplete, Skipped, etc.
- Screenshots or Video of Test Execution: Detailed information about the test execution, such as screenshots or video recordings.
- Error Information: Test logs provide the error messages, exceptions, stack traces, etc., for debugging purposes.
- Comments or Observations: Any additional observations or comments from the test executor can be added to help with the test execution results or any other information.
Steps to Use Test Logs
-
Choose a Suitable Tool
Use a suitable tool for setting up test logs based on your project need, size, and your team’s skills. You can use a basic tool such as MS Excel to set up test logs with Selenium scripts or use the Logger class. You can use Apache Log4j for a more advanced test log experience. However, then you must add Log4j dependencies and configurations in the test automation framework. Then, you need to code the test scripts to use this test log feature, which is an overhead and requires good programming skills.
As an in-built feature, better options that do it all for you are available. One such intelligent tool is testRigor, which creates test logs and reports with screenshots, error messages, and video recordings of the test execution inherently with the test runs. You need not to do anything. In later sections, we will see the ease and effortlessness of using testRigor.
-
Create a Test Log Template
If you are creating the test scripts yourself through coding, then choose the headers of the test log template as ‘Test Case ID’, ‘Description’, ‘Execution Date’, ‘Tester’, ‘Result’, ‘Comments’, and ‘Defect ID’ (if applicable). Otherwise, modern tools let you customize and provide the test log’s built-in features, saving you tons of time and effort in coding the reporting and test log parts.
-
Select Test Cases
Shortlist the test cases that you need to execute for that testing cycle. If you are using CI/CD pipelines, then the test cases should be integrated with them.
-
Execute Test Cases
Start the execution if you are manually testing the test suite. If test automation is involved, trigger the test run.
-
Record the Results and Failures
Record the date/time of execution, the tester’s name, and the results: Pass, Fail, Skip, Incomplete. You need to capture and save the screenshots yourself if you are using Selenium. If there are defects, mention the defects (ID and description) in the test logs. Provide any other references and comments if required.
Here are the 11 reasons why not to use Selenium for automation testing. Modern tools allow direct integration with defect management tools and automatically link test cases with the defect ID. If there are good options available, why not use them?
-
Save and Share
Make arrangements to save and share the test logs with developers, the testing team, and other stakeholders through coding if using Selenium. However, there are better options, one of which we have discussed in the next section.
testRigor for Easy and Quick Test Logs
If you want to bypass mentioning and coding the dependencies and configurations in your Selenium scripts, try some intelligent tools to save effort and time. testRigor is a generative AI-powered testing tool that lets you create test steps in plain English. No programming is involved. It saves you test maintenance time since it works on self-healing tests.
enter "TV" into "Search" enter enter click "TV" click "Add To Cart" check that page contains "Item Added To Cart"
As you can see in the screenshot below, testRigor saves the test pieces of evidence, such as screenshots, by default. Click ‘More Details’ to view the URL, console logs, session keys, etc.
Click on ‘View Execution’ to view all the screenshots, errors, test ID, test steps, results, execution date/ time, execution user, execution time spent, etc. If you wish to save it in Word or PDF files, you can easily do so in a few clicks.
Read here about the Test Reports – everything you need to get started.
Best Practices for Setting Up Test Logs
You can follow the best practices below to benefit from this test artifact.
Use Uniform Test Log Format
Maintain uniformity with your test logs using a standard template for recorded information, such as timestamps, test case identifiers, outcomes, error messages, etc.
Another critical element to remember is that the test logs should be clear and concise. Too much information is non-practical to handle.
Automation is the Key
Use logging commands or tools to incorporate automated logs into the test scripts. Otherwise, use modern tools such as testRigor, which provide these capabilities in-built. Capture relevant and concise data such as execution time, test parameters, pass/fail status, and error messages.
Capture Screenshots and Error Details
In case of errors, you should capture stack traces, error messages, and specific failure points. This information helps developers resolve defects/bugs quickly and efficiently because they know where to act. Visual evidence, such as screenshots and video recordings of the test execution, helps the developers to fix bugs effectively. Also, they serve as a piece of historical evidence to go back and check in case of any issue in the future.
Implement Log Levels in Test Scripts
If using programmed test scripts such as in Selenium, use log levels such as DEBUG, INFO, WARN, ERROR, etc. This categorization of log entries helps filter logs based on the required detail level.
Logs should be Accessible
What is the use of having test logs if they are not accessible and available when required? They should be present in a centralized repository for easy access. Also, you can use tools that provide filtering or searching of the log data.
Follow Compliance and Security
If the logs contain sensitive information, mask or encrypt sensitive data. Be aware of and comply with industry-specific log management and data retention regulations.
Use CI/CD Integration
Integrate test logging with your CI/CD pipeline to automatically generate reports after each build or test run. Use logging to provide real-time feedback to the development and QA teams to help them come up with quick responses to failures or issues.
Conclusion
The test log is used to have automated, structured, informative, and easily accessible information about the test execution handy. With CI/CD and Agile methodologies, there is no scope to waste time getting the relevant information. All the required information should be present on the tip of the fingers within clicks.
Here, the choice of test automation tools and the related ecosystem are critical players in testing success. Use intelligent AI-enabled testing tools such as testRigor so that you can use your focus only on creating robust test cases and not on setting up testing prerequisites such as test logs.
