Azure Testing

Hari Mahesh

Microsoft Azure’s initial focus was on web hosting and data storage, which is ideally suited for web developers. Azure’s offerings have grown tremendously since its inception in 2010. Today, it is a leading cloud service provider, rivaling tech titans like Amazon Web Services (AWS) and Google Cloud Platform (GCP).

The platform now boasts a robust suite of services that span across three key areas:

• Infrastructure as a Service (IaaS): This layer offers building blocks like virtual machines (VMs), storage, and networking resources. Imagine it as renting computing power, storage space, and the connections to link everything together. You have granular control over these resources to precisely tailor them to your needs.
• Platform as a Service (PaaS): This layer simplifies development and deployment. Think of it as a pre-built development environment in the cloud. You can focus on writing code without worrying about the underlying infrastructure – servers, databases, and other technicalities are handled by Azure.
• Software as a Service (SaaS): This layer provides on-demand access to popular Microsoft software applications like Office 365 and Dynamics 365. There is no need for installation or maintenance – just log in and get started.

Products and Services Offered by Azure

Microsoft Azure offers an extensive range of products and services designed to meet the needs of various businesses, from startups to large enterprises and governmental organizations. Its core offerings include:

• Azure Virtual Machines for scalable computing power
• Azure SQL Database for managed database services
• Azure Active Directory for identification and access management

For developers, Azure provides a suite of tools, including:

• Azure DevOps for CI/CD pipelines
• Azure Kubernetes Service (AKS) for container orchestration
• Azure Functions for serverless computing, enabling rapid application development and deployment

Data and analytics are also a significant part of Azure’s offerings, with services like:

• Azure Synapse Analytics for big data and advanced analytics
• Azure HDInsight for processing big data
• Azure Cosmos DB for globally distributed database services

Azure’s AI and machine learning capabilities are embodied in Azure Machine Learning and Azure Cognitive Services, offering pre-built and customizable AI services for applications.

Additionally, Azure provides comprehensive solutions for IoT, blockchain, security, and compliance, ensuring businesses can build, deploy, and manage a wide range of applications and systems in a secure, compliant cloud environment. This vast array of services allows organizations to innovate, scale, and achieve their digital transformation goals efficiently.

Azure Marketplace

The Azure Marketplace is an online store provided by Microsoft that offers a wide range of applications and services specifically designed for use in Microsoft Azure. This platform is a hub where Microsoft partners and independent software vendors (ISVs) can publish their applications and services, making it easier for Azure customers to find, try, and deploy certified and optimized software solutions to run on Azure.

The offerings in the Azure Marketplace cover a broad spectrum of needs, including but not limited to cloud applications, development and operations (DevOps) tools, infrastructure software, and business solutions. These can range from open-source software to specialized industry-specific applications, encompassing analytics, security, machine learning solutions, and more.

Integrating Azure Products into Your Business

Integrating Azure products into your business doesn’t require building everything from scratch. Microsoft provides a comprehensive “toolkit” with resources to simplify the process. Here’s how to leverage it effectively:

• Azure SDKs: They are vital to incorporating Azure cloud services’ power into your applications. Available in various programming languages you know and love, including .NET, Java, Python, JavaScript/TypeScript, Go, PHP, and Ruby, these SDKs take the complexity out of interacting with Azure.
• Azure CLI and PowerShell: The Azure Command-Line Interface (CLI) and Azure PowerShell are powerful tools for managing Azure resources directly from the command line or scripts. They are instrumental in automating the deployment and management of Azure services, allowing for efficient integration and continuous delivery.

Testing Azure Integrations

Rigorous testing is crucial after integrating an Azure product or developing an app for the Marketplace. A multi-layered approach ensures top-notch quality. While considering different functional testing levels, we can start with unit testing to verify individual code components function as intended. Then, move on to integration testing to confirm other parts of your solution work together seamlessly. Finally, end-to-end testing will simulate real-world usage and identify potential issues.

Beyond these core tests, consider non-functional testing techniques to evaluate performance, security, accessibility, and scalability. This comprehensive approach guarantees a robust and well-functioning integration or application ready to excel in Azure.

Unit Testing Azure Integrations

Unit testing is an essential first step in ensuring the quality of your Azure integrations. It involves testing individual units of code, typically functions or classes, in isolation from the rest of your application and the Azure environment.

What it Tests?

• Isolated Components: Unit tests focus on individual code units within your integration logic. These units could be functions, classes, or modules responsible for interacting with specific Azure services.
• Azure Service Interaction: The tests verify that these code units interact with Azure services as expected. This might involve checking if data is sent or retrieved correctly using Azure SDK methods or REST APIs.
• Mocking Dependencies: Since unit tests aim to isolate the code being tested, external dependencies like Azure resources are often mocked. Mocks simulate the behavior of these dependencies, allowing you to test your code independently without requiring a live Azure environment.

Benefits of Unit Testing Azure Integrations

• Early Bug Detection: Unit tests help identify issues within your integration code early in development. This allows for faster fixes and prevents bugs from cascading into later stages.
• Improved Code Maintainability: Well-written unit tests act as documentation for your code. They explain how different parts of your integration logic are supposed to function, making it easier for you and others to understand and maintain the code in the future.

Unit Testing Tools

Several tools are available for unit testing Azure integrations, encompassing Azure-specific offerings and popular frameworks for various programming languages. The best choice of tools depends on your specific needs and preferences. If you’re already familiar with a particular testing framework for your programming language, consider leveraging it with the Azure SDK’s testing libraries for mocking functionalities.

• Azure SDK Test Libraries: Most Azure SDKs have built-in testing libraries. These libraries offer functionalities explicitly for mocking Azure services and writing unit tests for code interacting with those services. For example, the Azure SDK for Java includes the azure-identity-junit5 library for mocking authentication services during unit testing.
• JUnit: The industry standard for unit testing in Java. It provides many features, including assertions, test runners, and annotations for organizing tests.
• Pytest: A popular and versatile testing framework focusing on simplicity and readability. Offers features like test discovery, fixtures for test setup and teardown, and parametrization for running tests with different data sets.
• Jest: A robust testing framework offering features like test watching, code coverage reporting, and mocking capabilities. They are often used with popular JavaScript libraries like React and Node.js.

In addition to the testing frameworks, you’ll likely need mocking libraries to simulate the behavior of Azure services during unit tests. Popular options include:

• Mockito (Java): A widely used mocking library for creating mock objects that mimic the behavior of natural objects.
• Mock (Python): Built-in mocking library in the Python standard library, offering a simple approach to creating mock objects.
• Sinon.JS (JavaScript): A popular library for creating stubs, spies, and mocks in JavaScript unit tests.

Here is a sample unit testing code using Jest.

Let’s imagine we have a module that interacts with Azure Cosmos DB. We want to unit test a function named ‘addItem’ that adds an item to a Cosmos DB container, ensuring our tests run without actual calls to Azure Cosmos DB. We can write a Jest test for ‘addItem’ using mocking to simulate the interaction with Cosmos DB.

// cosmosDbService.test.js
const { addItem } = require('./cosmosDbService');

jest.mock('@azure/cosmos', () => {
  const mockContainer = {
    items: {
      create: jest.fn().mockResolvedValue({ resource: { id: '123', content: 'Test item' } })
    }
  };

  const mockDatabase = { container: jest.fn().mockReturnValue(mockContainer) };
  const mockCosmosClient = { database: jest.fn().mockReturnValue(mockDatabase) };

  return { CosmosClient: jest.fn().mockImplementation(() => mockCosmosClient) };
});

describe('addItem to Cosmos DB', () => {

  it('should add an item and return the added item', async () => {

    const item = { id: '123', content: 'Test item' };
    const addedItem = await addItem(item);
    expect(addedItem).toEqual(item);

    const { CosmosClient } = require('@azure/cosmos');
    expect(CosmosClient).toHaveBeenCalled();

    const client = new CosmosClient();
    expect(client.database).toHaveBeenCalledWith('YourDatabaseId');
    expect(client.database('YourDatabaseId').container).toHaveBeenCalledWith('YourContainerId');
    expect(client.database('YourDatabaseId').container('YourContainerId').items.create).toHaveBeenCalledWith(item);
  });
});

Here, we did the following two tasks:

• We mock @azure/cosmos entirely to intercept calls to the Cosmos DB client, simulating its behavior.
• The test verifies that ‘addItem’ correctly interacts with the mocked Cosmos DB client by expecting the added item to match the input item and checking that all relevant Cosmos DB client methods are called as expected.

Integration Testing Azure

Integration testing takes unit testing a step further, focusing on how different components of your Azure integration work together. In the context of Azure, this means verifying that your code interacts flawlessly with Azure services and other systems involved in the integration process.

What it Tests?

• Interaction Between Components: Integration tests ensure that various parts of your Azure integration, including custom code, Azure services, and external systems, communicate and exchange data effectively.
• Data Flow and Logic: These tests verify that data flows seamlessly between different components and that the overall integration logic functions as intended. This might involve testing if data is transformed correctly between systems or if Azure services interact with each other as expected.
• Error Handling: Integration tests should also assess how your integration handles errors or unexpected situations. This ensures the system behaves gracefully when encountering issues and doesn’t produce unpredictable results.

Benefits of Integration Testing Azure Integrations

• Early Detection of Integration Issues: Integration testing helps identify problems with how different components collaborate before deployment. This allows for early fixes and prevents issues from impacting production environments.
• Improved System Reliability: Integration testing contributes to a more reliable overall system by verifying proper communication and data flow. This translates to a more robust Azure integration that functions consistently as designed.
• Reduced Risk of Regression: When making changes to your integration, integration tests act as a safety net. Running these tests after modifications helps ensure the changes haven’t unintentionally affected the interaction between components.

Integration Testing Tools

• Azure Resource Manager (ARM) Templates: These templates define your Azure infrastructure and configuration. Use them to create dedicated test environments that mirror your production setup, allowing you to test your integration without impacting live resources.
• Azure DevOps Pipelines: Microsoft’s continuous integration and delivery (CI/CD) platform integrates seamlessly with Azure services. Utilize pipelines to automate your integration tests, streamlining the testing process and enabling continuous integration of code changes.

We can perform integration testing at the API and UI levels. So, let’s look at a few commonly used tools for performing integration testing for Azure integrations.

• API Testing Tools

• • Postman: This popular tool allows you to send requests to Azure service APIs, simulating real-world interactions. Postman helps verify how your code interacts with Azure services and identify API-related issues.
  • REST Assured: A Java-specific library for testing RESTful APIs. It simplifies writing automated tests for your Azure service interactions.

Regarding UI integration testing, the commonly used tool is Selenium, which we can discuss in E2E testing.

E2E Testing for Azure Integrations

For Azure integrations, end-to-end (E2E) testing goes beyond individual parts. It simulates real user scenarios, putting your entire system through the paces. This comprehensive approach validates how data flows and interacts between various Azure services – databases, storage, functions, and web apps. Imagine testing a user journey from start to finish, mimicking production conditions. E2E testing uncovers issues that unit or integration tests might miss, like data flow problems, service hiccups, or performance bottlenecks. Proactively identifying these issues ensures a seamless user experience and a more reliable, stable solution for your Azure deployments.

One of the commonly used e2e testing tools is Selenium.

Selenium

As an open-source tool, Selenium has been widely used by many teams. Selenium provides minimal integrations with other tools, but many third-party integrations can be plugged in with Selenium. Though there are a few advantages to using Selenium, there are many factors that were concerning while using Selenium, like:

• Maintenance Challenges: Selenium scripts can become complex and difficult to maintain, especially as your application evolves. Keeping tests up-to-date with frequent UI changes can be a burden.
• Limited Mobile and Desktop Application Support: While there are workarounds, Selenium is primarily designed for web application testing. It has limited support for native mobile apps and desktop applications.
• Dependency on Browser Updates: Selenium relies on browser behavior, and changes in browser versions can sometimes break your tests. You must update your tests to stay compatible with the latest browsers.
• Slow Script Execution: Selenium scripts can be slower than alternative testing tools. This can be a drawback for large test suites or performance testing.
• Lack of Reliable Technical Support: Selenium is an open-source project, and there’s no guaranteed official technical support available.

Above are a few reasons why Selenium is no longer used for automation testing. Here, we are not covering much detail about Selenium, but if you would like to know about building Selenium tests, you can refer to this blog.

testRigor for E2E Testing

As we know, many codeless automation tools are available in the market. Still, testRigor distinguishes itself through a combination of features designed to streamline the testing process and empower a broader range of users.

Let’s check what makes testRigor a standout choice:

• Effortless Setup: Forget time-consuming infrastructure setup and installation hassles. testRigor is a cloud-hosted solution that allows you to sign up, purchase a plan, and start testing immediately. This translates to significant cost and time savings compared to traditional tools.
• Integration Arsenal: No more scrambling to find compatible integrations after installation. testRigor boasts a comprehensive library of pre-built integrations, eliminating compatibility headaches and ensuring smooth operation even during upgrades. You can find a detailed list of supported integrations here.
• Plain English Power: testRigor breaks the barrier of programming expertise. You can create test scripts using natural language, similar to describing test steps in plain English. This empowers manual testers to create automated scripts faster than traditional automation tools engineers use. Additionally, business analysts and stakeholders can easily understand and modify these scripts, fostering better collaboration.
• Generative AI Boost: Leverage the power of generative AI to streamline test creation. Simply provide a description, and testRigor can generate test cases or test data automatically. This reduces manual effort and ensures your test suite covers a broader range of scenarios.
• Say Goodbye to Flaky Locators: Unlike traditional tools that depend on fragile XPath or CSS selectors, testRigor simplifies element identification for a more human-like experience. You simply describe the element you want to interact with by mentioning the text you see on the screen or its relative position through testRigor locators.
  Here is an example where you identify elements with the text you see for them on the screen.

  click "cart"
  click on button "Delete" below "Section Name"

• Unified Testing Platform: Eliminate the hassle of managing multiple testing frameworks for web, mobile, and desktop applications. testRigor acts as a comprehensive testing powerhouse, supporting a wide range of testing needs within a single platform. This simplifies your testing process and reduces tool sprawl and associated maintenance overhead. You can perform the below testing types using plain English commands:
  • Web and Mobile browser testing
  • Mobile testing
  • Desktop app testing
  • API testing
  • Visual testing
  • Accessibility testing

If you are excited to learn more about the features of testRigor, you can visit this page.

open url "https://www.loginurl.com"
enter stored value "UserName" into "User Name"
enter stored value "password" into "Password"
click "Sign in"
click “Employees”
click “Add New Employee”
enter "John" into "First Name"
enter "Doe" into "Last Name"
click "Submit"

The example above clearly shows the simplicity of testRigor test script creation, maintenance, and other test activities as automation progresses to the next level.

Azure Marketplace App Certification Process

Publishing an application to the Azure Marketplace involves a review process to ensure it meets Microsoft’s quality and security standards. Here’s a breakdown of the key steps:

Submission and Validation

• You submit your application details and package to the Azure Marketplace through the Cloud Partner Portal.
• Microsoft validates your marketplace registration information for completeness.

Technical Review

The Microsoft review team thoroughly evaluates your application for the following:

• Functionality: Does it work as intended and deliver the promised value?
• Compatibility: Does it integrate seamlessly with Azure services?
• Security: Does it adhere to Microsoft’s security best practices and industry standards?
• Performance: Does it perform efficiently and meet scalability requirements?
• Documentation: Is the documentation clear, concise, and helpful for users?

Feedback and Iteration

• You receive feedback from the review team outlining any issues identified.
• You have the opportunity to address these issues and resubmit your application.
• This iterative process continues until your application meets all the review criteria.

Approval and Publishing

• Once your application successfully passes the review process, it is published on the Azure Marketplace.
• You can then manage your listing, updates, and pricing through the Cloud Partner Portal.

References

Here are some helpful resources for further information on the Azure Marketplace app review process:

• Microsoft documentation: Review feedback for Azure app offers
• Azure Marketplace and AppSource publishing guide
• Microsoft blog: For building apps and services for Azure Marketplace and AppSource

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT