Using Jenkins for Automated Testing

Anushree Chatterjee

Imagine you’re part of a software development team, and every time a developer writes new code, someone has to manually run tests to ensure everything still works. This can be slow, error-prone, and exhausting, especially when the project grows.

Enter Jenkins, a tool that takes this repetitive task off your plate. Jenkins automates the testing process by running your tests as soon as new code is added. If a developer uploads a feature to the code repository, Jenkins can automatically trigger a set of tests to ensure the new feature doesn’t break existing functionality. It’s like having a reliable assistant who works tirelessly to keep your software in great shape and lets your team focus on building new features. That’s why so many teams trust Jenkins for automated testing.

Let’s learn more about Jenkins and how to use it for automated testing.

What is Jenkins?

So, what is Jenkins used for? Jenkins is an open-source tool that automates repetitive tasks in the software development process. It’s mainly used for CI/CD. These fancy terms simply mean that Jenkins helps developers merge their work frequently (Continuous Integration) and makes it easier to get the software ready to be delivered to users (Continuous Delivery).

For example, if you’re building an e-commerce site and a developer updates the payment system, Jenkins can immediately check that the update hasn’t broken the checkout process or other features.

How Jenkins Works

Think of Jenkins as a super-organized team member who takes care of repetitive and time-consuming tasks like running tests, building code, and deploying software. Here’s a simple explanation of how Jenkins works:

1. Jenkins Monitors Your Code: The Jenkins tool is connected to your code repository (like GitHub, GitLab, or Bitbucket). It’s always watching for changes. For example, if a developer adds a new feature or fixes a bug and pushes the updated code to the repository, Jenkins gets a notification about it.
2. Jenkins Gets to Work: Once Jenkins detects a change in the code, it springs into action and starts following the instructions you’ve given it (this is called a job or pipeline).
   Key Steps Jenkins Performs:
   • Pulls the Latest Code:
     • Jenkins downloads the latest version of the code from your repository.
     • Think of it as Jenkins grabbing the newest instructions for what to do next.
   • Builds the Code:
     • If your project needs to be compiled (like Java or C++), Jenkins builds the code.
     • For example, it might use tools like Maven or Gradle to turn your code into a runnable application.
   • Runs Automated Tests:
     • Jenkins runs the tests you’ve set up to check if the new code works and doesn’t break existing features.
     • For example, it might run unit tests (to test individual pieces of code) or UI tests (to test how your app looks and works).
   • Generates Reports:
     • After running the tests, Jenkins collects the results and generates a report showing what passed, what failed, and why.
3. Jenkins Shows You the Results: Once Jenkins finishes its tasks, it updates you with the results.
   • You can see:
     • Whether the build was successful or failed.
     • Which tests passed and which didn’t.
     • Logs and error details to help you fix any issues.
   • If configured, Jenkins can also send alerts via email, Slack, or other tools to notify your team immediately.
4. Jenkins Can Automate the Next Steps: If the tests pass, Jenkins can move on to the next steps automatically, like:
   • Packaging the software into a deployable format.
   • Deploying it to a test or production server.
   • Running additional performance or security tests.
5. Jenkins Uses Pipelines to Manage Workflows: Jenkins organizes these steps using something called a pipeline. A pipeline is like a recipe that tells Jenkins:
   • What tasks to do.
   • In what order to do them.
   • What to do if something goes wrong.
   For example:
   • Step 1: Check the code for updates.
   • Step 2: Build the code.
   • Step 3: Run tests.
   • Step 4: Package the code if all tests pass.
   • Step 5: Deploy to the server.
6. Jenkins Can Work with Multiple Machines
   • If the workload is heavy (like running a lot of tests), Jenkins can distribute the tasks across multiple machines, called agents or nodes.
   • This means Jenkins can test your app on different platforms (e.g., Windows, macOS, Linux) or handle multiple projects simultaneously.
7. Jenkins Is Flexible and Customizable
   • The Jenkins tool doesn’t work alone. It has thousands of plugins to connect with tools you’re already using, such as testing frameworks (JUnit), version control systems (GitHub, Bitbucket), or deployment tools (Docker, Kubernetes).
   • You can customize Jenkins to suit your exact needs.

An Example to Tie It All Together

Let’s say you’re developing a mobile app. Here’s how Jenkins works in action:

• A developer adds a new feature and pushes the code to GitHub.
• Jenkins notices the change and:
  • Pulls the updated code.
  • Builds the app to make sure it compiles.
  • Runs automated tests to check if the new feature works and doesn’t break anything else.
  • Sends a report to the team: “Build passed, all tests green!”
• If everything is fine, Jenkins packages the app and deploys it to a test server for further testing.

Key Features of Jenkins Relevant to Automated Testing

Jenkins is packed with features that make automated testing easier, faster, and more reliable. Let’s break down the most important ones:

Plugins Ecosystem

Jenkins is like a toolbox where you can add tools based on what you need. These tools are called plugins, and there are thousands of them available for Jenkins.

What do plugins do?

Plugins extend Jenkins’ capabilities. For example:

• If you need to generate test reports, you can add a plugin for JUnit or TestNG.
• Want notifications on Slack or email about test results? There are plugins for that too!

Why is this helpful for testing?

No matter what testing framework or tool you use, whether it’s for unit testing, UI testing, or performance testing, there’s probably a plugin that makes it work smoothly with Jenkins. This flexibility allows you to build a testing setup tailored to your needs.

Pipelines as Code

Jenkins lets you write your automation processes as code.

What is a pipeline?

A pipeline is a step-by-step process for building, testing, and deploying your software. For example, a simple pipeline might:

• Pull the latest code from the repository.
• Run automated tests to check the code.
• Package the code if all tests pass.

How does “as code” help?

Instead of clicking buttons in Jenkins to set this up, you write it down in a script. This script:

• Makes your testing process repeatable and consistent.
• It is easy to control the version, just like your app’s code.
• It can be shared across your team so everyone knows exactly what’s happening.

Why is this helpful for testing?

With pipelines as code, you can create complex testing workflows like running tests in parallel, testing on different environments, or retrying failed tests automatically. It gives you complete control and flexibility.

Scalability with Distributed Builds

Sometimes, testing can take a long time, especially if you have a lot of tests or need to run them on multiple platforms. This is where Jenkins’ ability to scale comes into play.

What does scalability mean here?

Jenkins can distribute the workload across multiple machines, which are called nodes or agents. Instead of running all your tests on one machine, Jenkins can run them on several machines at the same time.

How does it work?

Jenkins has a master-agent setup:

• The master coordinates the entire process – what tests to run, where to run them, and when.
• The agents do the heavy lifting by running the tests.

Why is this helpful for testing?

• Faster execution: By running tests in parallel across multiple agents, you save a lot of time.
• Test on different platforms: Need to test your app on Windows, Linux, and macOS? Assign agents with different operating systems.
• Handle large workloads: As your project grows, you can add more agents to handle the increasing number of tests.

Steps for Using Jenkins for Test Automation

Here’s a step-by-step guide to help you use Jenkins for automated testing.

1. Install Jenkins: You need to set up Jenkins on your system or server to use it. Download and install Jenkins on your server.
2. Set up Workspace: Create a dedicated workspace for your project within Jenkins. This is where your source code will be checked out and builds will be executed.
3. Install Necessary Software: Install all required software on the Jenkins server and any agent nodes like:
   • Java Development Kit (JDK): If your tests are written in Java.
   • Python: If your tests are written in Python.
   • Node.js: If your tests are written in JavaScript.
   • Testing Frameworks: Install the specific testing framework you are using (e.g., JUnit, NUnit, TestNG, pytest, Mocha).
   • Other Dependencies: Install any other required libraries or tools (e.g., Maven, Gradle).
4. Create a Jenkins Job: This is where you tell Jenkins what tasks to automate. For that, you need to:
   • Create New Item: In the Jenkins dashboard, create a new item (e.g., “Freestyle project” or “Pipeline”).
   • Configure Source Code Management:
     • Specify VCS: Connect your Jenkins job to your version control system (e.g., Git).
     • Provide Credentials: Configure credentials to access your repository.
   • Build Triggers: Define how your job should be triggered:
     • Poll SCM: Schedule periodic checks for changes in the repository.
     • Webhooks: Trigger builds automatically when changes are pushed to the repository (requires setting up webhooks in your VCS).
5. Define Build Steps/Add Build Steps: This is where you define the sequence of actions that Jenkins will perform during each build:
   • Checkout Code: Check out the latest version of your code from the repository.
   • Build Project: Compile the source code if necessary (e.g., using Maven or Gradle).
   • Run Tests: Execute your test suite using the chosen testing framework.
   • Publish Test Results: Publish the test results in a readable format (e.g., JUnit XML, HTML reports).
   • Archive Artifacts: Archive important artifacts (e.g., build files, test reports) for later retrieval.
6. Configure Post-build Actions
   • Send Notifications: Configure Jenkins to send notifications (e.g., email, Slack) to relevant stakeholders about build results.
   • Trigger Other Jobs: Trigger downstream jobs (e.g., deployment jobs) based on the success or failure of the current job.
7. Test and Refine
   • Run a Test Build: Trigger a build manually to test the job configuration.
   • Analyze Results: Review the build logs and test results to identify and fix any issues.
   • Refine Configuration: Adjust the job configuration as needed to improve build times, stability, and overall efficiency.
8. Continuous Integration
   • Monitor Builds: Continuously monitor the build status and test results in the Jenkins dashboard.
   • Address Issues Promptly: Address any build failures or test failures promptly to prevent issues from accumulating.
   • Regularly Update: Regularly update Jenkins, plugins, and dependencies to ensure stability and access to the latest features.

AI-powered Test Automation With Jenkins

Jenkins tool is the perfect assistant to help you automate various aspects of software development. It lets you add steps to your pipeline, which it systematically executes. You can further boost its efficiency by integrating Jenkins with a powerful test automation tool to make the automated testing step more efficient.

Integrating Jenkins With testRigor

AI-powered tools like testRigor help automate end-to-end testing smartly. This tool uses generative AI to allow easy test creation and maintenance. Anyone on your team can write test cases with testRigor since it requires writing them in plain English language, unlike many other tools that require one to write code. These test cases can run across platforms and browsers and cover a wide range of scenarios, thanks to testRigor’s powerful command library.

You can integrate your testRigor test cases with Jenkins by using the pre-generated scripts in the CI/CD Integration section within a test suite. Here’s a detailed guide on how to integrate testRigor with Jenkins.

This tool brings down test maintenance to a bare minimum by using AI to generate ultra-stable test runs and handle most of the changes that might occur on the application screen. This is possible as testRigor does not depend on implementation details of UI elements like XPaths or CSS. Instead, it identifies these elements like buttons and UI fields the way a human would, by their appearance on the screen.

Imagine how smooth your testing process would be if you combined the power of testRigor with Jenkins!

That’s not all there is to testRigor. You can do a lot more with this tool. Take a look at the full feature list over here.

Conclusion

A process automator like Jenkins can make your life much easier. It can watch your code, run your tests, and help you deliver high-quality software faster and with less effort. All you have to do is tell Jenkins what you want it to do, and it takes care of the rest. Pair it with powerful tools, and you should be able to deliver high-quality software with ease.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT