CI/CD Series: testRigor and Azure DevOps
 |
TestRigor manages different types of CI/CD tools in order to enhance with AI your workflows and pipelines. This time, we’ll explore one of them: Azure DevOps.
Azure DevOps Brief Concept:
Azure DevOps is a cloud-based platform for DevOps lifecycle management, offering CI/CD pipelines, source control (via Git), and project management tools. Compared to GitHub Actions, which integrates deeply with GitHub for automation, Azure DevOps provides a more comprehensive suite for enterprise-scale projects with broader tool integration. GitLab CI/CD offers a similar CI/CD experience but is tightly integrated into GitLab’s platform. Azure DevOps stands out for its flexibility, supporting multiple version control systems (Git and TFVC), and extensive services, making it ideal for large organizations managing complex workflows.
Azure DevOps: Tools and Considerations:
-
Migration Tools: There’s no direct, automated tool to migrate workflows from, so manual conversion is needed.
-
Documentation: Microsoft and GitHub both provide comprehensive documentation on YAML syntax and how to build workflows in each system, which is helpful during the transition.
Comparison to other Platform Concepts:
-
Jobs and Steps: Jobs correspond to Azure Pipelines stages and jobs.
-
Triggers: It translates to Azure’s trigger: (for branches, PRs, etc.).
-
Marketplace: Azure DevOps has extensions in its own Marketplace for different languages.
How does testRigor work with Azure DevOps?
Seeing previous differences, testRigor smooths the testing process in the CI cycle when using Azure DevOps by adding a PowerShell Script that you can add in order to automate your Tests. You can find this prepopulated CI Script by going to testRigor workspace -> CI/CD Integration and click on PowerShell variation next to Bash option:
Caption: CICD powershell location
Each one of the current APIs comes with a PowerShell Script for it. Now, here we share the steps to run a single Azure DevOps Pipeline with TestRigor:
Before starting:
-
You already have a testRigor Workspace
-
You already have an Azure DevOps account and workspace.
-
You already performed the Azure DevOps testRigor Integration.
Steps
Step1: Login to Azure DevOps and go to your workspace.
Caption: Azure DevOps organization location
Step 2: You chose your organization or create a new one. For this tutorial, I will focus on an organization that already exists:
Caption: Organizations in Azure Account
Step 3: In the previous image, click on “Pipelines”. You will see there all your previous and current workflows. We will select a new pipeline for this:
Caption: Pipelines location
Step 4: Once there, and in the case it’s a new repo, you will have the option to:
-
Import from Azure Repo
-
Import from BitBucket
-
Import from GitHub
If you already performed the GitHub Actions and TestRigor CI: How it works article, it’s very easy, cause the only thing you require to perform is to transfer the current code from Github, update the .yml file to be with PowerShell scripts, run it and that’s it.
Caption: Import from GitHub to Azure
Step 5: So, if you did the previous steps to import from GitHub to AzureDevOps, convert the .yml file from using Bash to use the PowerShell APIs. However, if you imported the repo directly in Azure DevOps, it’s fine, because the only thing different is that we’ll add a .yml file that will help us to execute our Pipeline with testRigor. No matter if you imported or created a new repo, the repo structure will be the same, which is this one:
trigger:
branches:
include:
- main # Trigger on changes to the 'main' branch
pool:
vmImage: 'windows-latest' # Using a Windows agent
jobs:
- job: PowershellJob
displayName: "Run PowerShell Commands"
steps:
- task: PowerShell@2
displayName: "Download yq and Run Script"
inputs:
targetType: 'inline'
script: |
# Download yq binary for Windows
Invoke-WebRequest -Uri "https://github.com/mikefarah/yq/releases/latest/download/yq_windows_amd64.exe" -OutFile "$env:USERPROFILE\yq.exe"
Write-Output "yq downloaded successfully"
# Add yq to PATH for this session
$env:PATH += ";$env:USERPROFILE"
# Set headers and initialize JSON base
$headers = @{
"auth-token" = @{auth-token};
}
$jsonBase = @{
"baselineMutations" = @();
}
# Directory containing YAML test cases
$yamlFolder = "test-cases"
# Iterate over each YAML file, converting it to JSON and adding it to the array
foreach ($yamlFile in Get-ChildItem -Path $yamlFolder -Filter *.yaml) {
Write-Output "Processing file: $($yamlFile.FullName)"
# Convert YAML to JSON using yq
$jsonContent = & "$env:USERPROFILE\yq.exe" eval -o=json $yamlFile.FullName
Write-Output "Parsed JSON content: $jsonContent"
# Parse JSON content into an object if it’s not empty
if ($jsonContent) {
# Convert JSON content to PSCustomObject
$jsonContentObj = $jsonContent | ConvertFrom-Json
# Check and add the description property
$fileName = [System.IO.Path]::GetFileNameWithoutExtension($yamlFile.Name)
$jsonContentObj | Add-Member -MemberType NoteProperty -Name "description" -Value $fileName -Force
# Verify that all required fields are present
if (-not $jsonContentObj.customSteps) {
Write-Output "Warning: 'customSteps' field is missing in $fileName"
$jsonContentObj | Add-Member -MemberType NoteProperty -Name "customSteps" -Value "default steps" -Force
}
if (-not $jsonContentObj.labels) {
Write-Output "Warning: 'labels' field is missing in $fileName"
$jsonContentObj | Add-Member -MemberType NoteProperty -Name "labels" -Value @() -Force
}
# Append the JSON object to baselineMutations
$jsonBase.baselineMutations += $jsonContentObj
Write-Output "Successfully added description to JSON object"
} else {
Write-Output "Error: Failed to parse JSON content from $yamlFile."
exit 1
}
}
# Convert final JSON to string
$body = $jsonBase | ConvertTo-Json -Depth 10
Write-Output "Print body: $body"
try {
# Make API POST request
Invoke-RestMethod -Method POST -Headers $headers -ContentType "application/json" -Body $body -Uri "https://api.testrigor.com/api/v1/apps/@{your-testSuite}/retest" -ErrorVariable myerror
} catch {
# Handle errors
Write-Output "Error calling API"
Write-Output $_.Exception
exit 1
}
Start-Sleep -Seconds 10
# Begin polling to check retest status
while ($true) {
Write-Output "==================================="
Write-Output " Checking TestRigor retest status"
Write-Output "==================================="
try {
$response = Invoke-WebRequest -Method GET -Headers $headers -Uri "https://api.testrigor.com/api/v1/apps/eiRFH5aYEDdSjrTku/status" -UseBasicParsing -ErrorVariable myerror
} catch {
Write-Output "Error calling API"
Write-Output $_.Exception
exit 1
}
$code = $response.StatusCode
$body = $response.Content
Write-Output "Status code: $code"
Write-Output "Response: $body"
switch ($code) {
200 {
Write-Output "Test finished successfully"
exit 0
}
{ @(227, 228) -contains $code } {
Write-Output "Test is not finished yet"
}
230 {
Write-Output "Test finished but failed"
exit 1
}
default {
Write-Output "Unknown status: $code"
exit 1
}
}
Start-Sleep -Seconds 10
}
IMPORTANT: The @{auth-token} is the token found in the CI/CD section, and the @{your-testSuite} is the test suite Id. We added a .env file to grab them from there due to security reasons, so replace them with your own token and test suite Id in order to make it work properly.
Step 6: Now, we’ll proceed to set up a new build by selecting our pre-existing .yml file:
Caption: Pipelines location
Caption: Existing Azure YAML file selection
Step 7: Select your branch:
Caption: Select .yml file and branch location
Step 8: We review the pipeline and run it:
Caption: Save and Run Button
Caption: Commit Message
Step 9: We review the current workload and status and that’s it:
Caption: TestRigor build 1
Caption: TestRigor build 2
Caption: TestRigor build 3
And this is how it would look like in TestRigor:
Caption: TestRigor Run from Azure DevOps
