How to Increase Engineering Output Using AI

Pragya Yadav

“Once you have mastered time, you will understand how true it is that most people overestimate what they can accomplish in a year and underestimate what they can achieve in a decade!” –Tony Robbins.

When discussing output or productivity in software engineering, that can not always be in functions of time or lines of code. You may write thousands of lines of code quickly, but if those programs fail to fulfill customers’ expectations, it wastes time and effort. It is similar to the fact that the speed of a train might be lightning fast, but if it is on the wrong track, leading to a wrong destination, it is not worth it!

This article will discuss engineering output, the related KPIs, and, with examples, how to increase engineering output using AI and SDD (Specification-Driven Development). We will also look at tools and books that might help.

What is Engineering Productivity?

Engineering output in the context of a software company refers to the quantity and quality of work produced by the engineering team. The most important thing to remember is that it includes both the visible and the invisible output, such as customer satisfaction, impact, and value of your work.

The deliverables below are the engineering output, and there could also be more additions.

• Code: The primary output consists of new features, bug fixes, and enhancements to existing software.
• Documentation: Comprehensive and clear documentation for both internal developers and external users, including API documentation, user manuals, and in-code comments.
• Software Designs and Architectures: High-level designs and architectural plans that outline the structure and behavior of software systems.
• Tests: Automated and manual testing efforts ensure the software meets the intended quality standards and functions.
• Deployments: The process of releasing new or updated software versions to production environments, making them available to end-users.
• Tools and Utilities: Development of internal tools that increase efficiency, such as deployment scripts, testing frameworks, or any utility that aids in the software development process.
• Technical Solutions: Innovative technical solutions to complex problems, which may include research and development of new technologies or algorithms.

What Factors Affect Engineering Productivity?

• Quality: The reliability, maintainability, and performance of the software produced.
• Timeliness: The speed with which new features, bug fixes, and updates are delivered.
• Innovation: The introduction of new ideas, methods, or products that improve the software or the development process.
• Efficiency: The optimal use of resources (time, effort, cost) to achieve the desired outcomes.

You can measure engineering productivity and then, based on the numbers, improve and change processes. The team should recognize the customer satisfaction the product will bring and its business value; this is also known as the Business Value Framework. Read: How to Align Software Engineering with Business Goals.

KPIs to Measure Engineering Output

You can measure engineering productivity with basic KPIs such as lines of bug-free code, the number of features released in a release, the number of stories/features completed in the last sprint, etc. However, they should ideally indicate the team’s efficiency, quality of work, and contribution to business goals. Here is a way to measure how much effort by the product team will result in how much business/customer value.

We can use the Business Value Matrix for this purpose, as shown below. We must choose tasks that require little effort and provide high value. Brainstorm about the outcomes to be achieved, the capacity of the product team, the associated business/customer value, and finally, the tasks that will lead to these outcomes.

The other KPIs are as follows, which might help you evaluate and improve the engineering productivity:

Code Quality Metrics

• Defect Density: Measures the number of defects per thousand lines of code, giving an insight into the quality of the codebase. Read this article for Mastering Defect Management.
• Code Coverage: The percentage of the codebase tested by automated tests, indicating the potential risk of undetected bugs.
• Technical Debt Ratio: The proportion of time spent addressing technical debt versus adding new features indicates the health of the codebase. Technical debt represents the future cost of fixing or refactoring hastily written or poorly designed code. Just like financial debt, technical debt accumulates interest, meaning the longer it remains unaddressed, the more time and resources it will require to fix later.

Example: Microsoft – Code Health/Quality

Microsoft strongly emphasizes maintaining the health of its codebase, using metrics like bug resolution time and defect density to ensure high quality. They have developed internal tools to automate code reviews and testing, focusing on keeping technical debt low and code quality high. By prioritizing code health, Microsoft ensures that its vast array of software products remains reliable and maintainable over time.

Productivity Metrics

• Velocity: The amount of work a team completes in a sprint (or other set timeline). It is usually measured in story points or issue counts. It helps measure team speed and estimate future delivery timelines.
• Commit Volume: The number of commits made to a repository over time, indicating the activity level and progress.
• Feature Delivery Rate: The rate at which new features or user stories are completed and delivered, reflecting the team’s ability to add value to the product.

Example: Google – OKRs (Objectives and Key Results)

Google is famous for using OKRs to set and communicate goals and results at the company, team, and individual levels. For engineering teams, specific key results can include metrics like defect density reduction, increasing code review efficiency, or improving feature delivery rate. This approach helps teams align their daily work with the company’s strategic goals and create a culture of accountability and continuous improvement. Read this article to know How to Setup OKRs Properly.

Operational Efficiency Metrics

• Lead Time: It is the time taken from a new feature request or bug report to its deployment in production. It measures the efficiency of the development and deployment pipeline.
• Cycle Time: It is the time required to complete a single task or user story from start to finish, indicating the team’s efficiency in handling tasks.
• Change Failure Rate: The percentage of deployments causing a production failure, reflecting the delivery process’s stability and reliability.

Example: Netflix – Operational Efficiency

Netflix is known for its high operational efficiency, ensuring its services are always available to millions of users worldwide. It uses metrics such as uptime/availability and change failure rate to measure the reliability of its deployments. By rigorously monitoring these KPIs, Netflix minimizes downtime and maintains a seamless viewing experience, even when deploying new features or updates.

Quality and Reliability Metrics

• Bug Resolution Time: The average time to resolve a bug indicates the team’s responsiveness to issues. Read: about Minimizing Risks: The Impact of Late Bug Detection.
• Uptime/Availability: The percentage of time the application is available and operational, which is critical for customer-facing services.
• Customer Satisfaction Scores: Feedback from users regarding their satisfaction with the product, which can include Net Promoter Score (NPS) or other survey metrics.

Example: Atlassian – Customer Feedback Loops

Atlassian, the company behind products like Jira and Confluence, heavily incorporates customer satisfaction scores into its development process. By continuously collecting and analyzing user feedback, the company aligns its product development efforts with customer needs, ensuring that features add real value. This customer-centric approach helps prioritize the backlog and guide development efforts efficiently.

Collaboration and Engagement Metrics

• Pull Request (PR) Size: This is the average size of pull requests, with smaller PRs often indicating better code modularity and easier code review processes.
• Code Review Time: The average time taken for code reviews, reflecting on the team’s collaboration and code quality control processes.
• Employee Engagement Scores: Surveys or assessments measuring the engagement and satisfaction of the engineering team, as motivated and engaged teams are generally more productive.

Example: GitHub – Open Source Contributions

Open-source projects on GitHub often use KPIs like commit volume, pull request size, and code review time to gauge community engagement and project health. High commit volume and frequent contributions indicate a vibrant and active project, while swift code review processes suggest a responsive and collaborative community. These metrics are crucial for maintaining an active and healthy open-source ecosystem.

How to Increase Engineering Output Using AI

Many customized processes can be adopted per the project’s need to increase engineering productivity. Here we mention a few of them:

Specification Driven Development

Specification Driven Development (SDD) provides tremendous benefits, drastically simplifying the whole flow and saving software teams a lot of time. In this approach, the team can write a test in plain English, which is also an executable specification. This means that once the development team is done building the feature, the test can be executed immediately without any additional modifications. This saves a tremendous amount of time, effort, and money.

click "Register"
generate by regex "[a-z0-9]{30}\@my_company\.com", then enter into "Email" and save as "newEmail"
enter "MyPassw0rd123" into "Password"
click "Sign Up"
check that email to saved value "newEmail" was delivered
check that page contains button "Confirm Registration"
click "Confirm Registration"
check that page contains text "Successfully confirmed"

AI’s Role: By understanding these simple specifications, AI can generate comprehensive test cases for an application. Also, AI can help to generate cover edge cases and scenarios that might not be immediately obvious to human testers.

Use tools to save time

‘Now‘ is a great time to be alive! Numerous tools help you in every step of software development. Great tooling enables you to save time and put it to better use. The saved time and mental energy can be utilized for more creativity and innovation. For example, in a polyglot project environment where many programming languages are used, tools can help everyone code, irrespective of their programming proficiency.

Tools: Visual Studio Code for coding, Git for version controlling, Jenkins for CI/CD, SonarQube for code review, Trello/Asana for team collaboration, Confluence for documentation, Docker/Kubernetes for containerization, JIRA for project management, TestRail for test management, testRigor for automation testing, New Relic/Datadog for monitoring, are few examples of the supportive tools. Read: Top 7 CI/CD Tools to Explore in 2024 and Top 7 Automation Testing Tools to Consider.

AI’s Role: AI can automate routine coding tasks, offer code suggestions, and improve code quality, reducing the time developers spend on these tasks. Tools like Visual Studio Code can integrate AI to provide smarter code completion, while CI/CD tools like Jenkins can use AI to predict and mitigate potential deployment issues, optimizing the development pipeline.

Pre-built components

If you are building it from scratch, you are definitely doing it wrong. Use lots of pre-built components to create code faster and with minimal errors. They increase engineering output by providing ready-made solutions for common problems. These components can range from frontend UI elements to backend functionalities, offering a quick way to add complex features without building them from scratch.

Tools: Bootstrap/Material-UI for the frontend, Spring Boot/ Django for the backend, Auth0 for authorization and authentication, Stripe/ Paypal for payment processing, GitHub/GitLab for code management, Google Analytics for customer insights, etc., are a few examples of pre-built components.

AI’s Role: AI can help developers find the most suitable pre-built components for their needs by analyzing project requirements and suggesting the best matches. This can drastically reduce development time and increase the reliability of the software being developed.

Intelligent and efficient tools for QA

Your testing tools should reduce your burden and not increase the time spent on coding/debugging/maintenance. Use intelligent generative AI-driven tools such as testRigor to create end-to-end tests quickly in plain English and seamlessly integrate with CI/CD pipelines, test management tools, and infrastructure providers for smooth testing cycles. It lets you automate complex scenarios such as 2FA, QR codes, captcha, email, SMS, table handling, phone calls, files, audio, database, data-driven, geolocation testing, etc, in plain English test scripts.

AI’s Role: AI-driven tools can revolutionize QA by automating test case generation, identifying potential areas of failure, and optimizing test coverage based on past data. Tools like testRigor enable writing tests in plain English, making test automation accessible to team members not specialized in QA. Thus, they streamline the testing process and integrate it easily into CI/CD pipelines. Read to know What is Continuous Testing?

Product mindset and culture

Everyone on the team should view and build from the customer’s perspective. The question, “Why are we doing it?” should be frequently asked. The mindset should be shifted from building software to empowering the customers. The goal should not be to create the best technology or code but the best customer product. For example, if some team member thinks after noticing a UI bug, I am working on the application’s backend, and this is a front-end problem that I do not own. This is an incorrect approach. Here, everyone should cultivate a shared mindset that the whole team owns the product that is being built. And this creates magic!

To fix this, ask the team to track customer usage/reviews of the features that the engineers have built themselves and then work on improvements. This will allow them to feel a sense of responsibility and ownership toward the feature they are creating and redirect them to think from the customer’s perspective.

AI’s Role: AI analytics and customer insight tools can provide real-time feedback on how users interact with a product, identifying both pain points and areas of success. This allows engineering teams to iterate more effectively, focusing on features and fixes that directly enhance user experience.

Retrospective meetings

Introspect and retrospect: What can go wrong, and how can we prepare for it within the team? Depending on your development cycle, retrospectives can be held at the end of each sprint (for Agile teams) or at the end of a project/milestone. They should include the entire team, developers, testers, project managers, and any stakeholders willing to participate.

Tools: Use retrospective tools like FunRetro, Retrium, or simply a shared document to facilitate remote or in-person retrospectives.

Examples: Ask what practices the team should start doing, stop doing, and continue doing to improve output. Adopt ‘Five Whys’: When a problem is identified, ask “why” five times to get to the root cause.

AI’s Role: AI-driven analytics can help identify patterns in team performance and project outcomes, offering insights for retrospectives. Tools that facilitate retrospectives can use AI to suggest discussion points based on project data, helping teams focus on the most impactful improvements.

Manage non-development time

Identify the instances where the engineers are not developing and how to reduce that non-development time. This often includes meetings, administrative tasks, bug fixing due to unclear requirements, and waiting for dependencies or feedback. Read: How to Save Budget on QA.

For example, pick-up time is when a developer or team gets up to speed on a new task, project, or technology. This is considered non-development time because it doesn’t directly contribute to creating code or immediate project deliverables. Pickup time includes activities like researching new technologies, understanding existing codebases, learning about the business domain, setting up development environments, and reading documentation. This pick time should be reduced to help engineers use that time more proactively and creatively.

Examples: Non-development time can be reduced through good documentation, setup automation scripts, reusable component libraries, allocating specific time for “spikes” to research new technologies or approaches, static code review tools, reducing PR size, reducing code review time, etc.

AI’s Role: AI can optimize schedules by identifying optimal times for meetings, suggesting periods for focused work based on individual productivity patterns, and automating routine administrative tasks. This minimizes downtime and maximizes the time engineers spend on development.

Use batching

Batching, a time management technique that involves grouping similar tasks together and tackling them in dedicated time blocks, can be effectively applied to various software development activities. This approach improves focus and reduces context switching and the cognitive load associated with shifting between different types of tasks.

Examples: Allocate dedicated time for reviews, such as the first hours of your workday. You can use tools or scripts to filter and prioritize PRs for review based on their age, the size of changes, or the criticality of the feature. If several small PRs touch the same parts of the codebase or relate to the same feature, consider reviewing them together. Another example is batch administrative tasks, such as email correspondence, filling timesheets, or updating task statuses in project management tools.

AI’s Role: AI can help organize and prioritize tasks, ensuring that similar tasks are grouped and addressed at optimal times. This can reduce context switching and improve focus, significantly enhancing productivity.

Good talent goes a long way

A point that is quite important is to have great talent on your team. It is incredible to have a team with the spirit of delivering a great product and bringing their A-game to work daily. While interviewing your team members, make sure that you look for the right mindset. Technical skills can be taught; however, the right mindset cannot.

AI’s Role: AI-driven HR tools can optimize the recruitment process by identifying candidates who not only match the technical requirements but also fit the company culture and team dynamics, ensuring a high-performing and harmonious team.

Pair programming

It is a technique where two developers work together at one workstation to write code, which is a core practice of Agile development methodologies. Despite initially seeming less efficient due to involving two developers in a single task, pair programming can significantly increase engineering output and overall code quality. With two sets of eyes on the code as it’s written, bugs and mistakes are often caught in real-time, reducing the need for extensive debugging and code reviews later. Pair programming ensures that more than one person is familiar with each part of the codebase, making it easier to manage absences or turnover.

Example: New team members can quickly become familiar with a project and the team’s coding standards through pair programming with an experienced developer.

AI’s Role: AI can support pair programming by recommending optimal pairings based on skill sets, experience, and even past project success. This ensures that the pairing is as productive as possible, enhancing both learning and output.

Minimize context switch

“What looks like multitasking is really switching back and forth between multiple tasks, which reduces productivity and increases mistakes by up to 50%” – Susan Cain.

Context switching involves shifting attention from one task to another, which can cost valuable time and mental energy as the developer needs to reload the context of the new task into their working memory. While reviewing, you see that the code is large and cannot be completed. So, you start doing something else, and then to start the same task the next day, you have to remember what you reviewed the previous day. Or if you are working on incorporating the review comments in code, then this context switch may result in thinking: What were the review comments? This is a sheer time waste.

Example: Implement a limit on the number of tasks or projects a developer or team is actively working on, i.e., work in progress (WIP). Agile methodologies like Kanban emphasize limiting WIP to improve focus and productivity. Another critical point is to have asynchronous communication, such as emails, message boards, and project management software, over real-time communication, like instant messaging and unscheduled calls.

AI’s Role: AI can manage task assignments and notifications to minimize unnecessary interruptions, using predictive algorithms to schedule tasks in a way that reduces the need for context switching. Here is A Roadmap to Better Agile Testing.

Balance the work

“Don’t worry about breaks every 20 minutes ruining your focus on a task. Contrary to what I might have guessed, taking regular breaks from mental tasks actually improves your creativity and productivity. Skipping breaks, on the other hand, leads to stress and fatigue“ – Tom Rath.

We do not have any metrics on it. Connect with people, talk to them, and identify if they are stressed. Rest, relax, and reflect: do not burn out. Take a break after every 60 minutes to use the full power of your brain and provide crisp performance.

AI’s Role: AI-driven project management tools can monitor team stress and workload, suggesting breaks and workload adjustments to prevent burnout and maintain high productivity.

Read how to perform Test Automation with AI and the Future of Testing.

Books to Read

• Accelerate by Nicole Forsgren, Jez Humble, Gene Kim
  Explores the impact of DevOps practices on software development and IT operations. It presents research-backed insights on how technological and organizational practices can accelerate the performance of technology organizations.
• The DevOps Handbook by Gene Kim, Jez Humble, Patrick Debois, and John Willis
  Offers a practical guide for implementing DevOps principles to achieve high velocity in technology organizations. Emphasizes automation, continuous delivery, and a culture of collaboration.
• The Lean Startup by Eric Ries
  Introduces a methodology for developing businesses and products. This methodology aims to shorten product development cycles and rapidly discover if a proposed business model is viable through iterative product releases and validated learning.
• Outcomes Over Output by Joshua Seiden
  Argues for focusing on customer behavior and outcomes rather than just output as the primary driver of business success. Advocates for a shift in how teams approach product development and measure progress.
• High Output Management by Andy Grove
  Provides timeless management principles and practices from the perspective of a legendary Silicon Valley leader. Focuses on creating highly productive teams and companies through rigorous planning, motivation, and organizational strategies.

Conclusion

To increase engineering output, it is essential to focus on strategies that enhance both the efficiency and effectiveness of development processes. This includes embracing practices such as DevOps for faster, more reliable delivery cycles, implementing Agile methodologies for flexibility and responsiveness, and leveraging lean principles to minimize waste and focus on value creation. Ultimately, increasing engineering output requires a balanced approach combining the right mix of processes, tools, and cultural practices tailored to each organization’s unique needs.

AI, automation, and supporting tools are crucial to achieving an excellent engineering output. Intelligent and efficient tools are available, and every team should make maximum use of them to achieve engineering goals faster, effectively, and efficiently.

Frequently Asked Questions (FAQs)

Read: Continuous Integration and Testing: Best Practices.

Focus on clear communication, set and prioritize goals, and use collaboration tools like Slack and Trello for efficient project management. Use asynchronous work methods to allow flexibility while minimizing disruptions. Also, encourage a solid work-life balance to prevent burnout and support a dedicated home office setup to reduce distractions.

“We spent so much time on maintenance when using Selenium, and we spend nearly zero time with maintenance using testRigor.”

Keith Powe VP Of Engineering - IDT