Engineering metrics can be misleading when they are viewed in isolation, but by pairing certain key metrics together, engineering leaders can unlock the full picture of their software development pipeline.
Incorporating metrics into your leadership strategy can help fine tune engineering processes, offering objective measurements to check assumptions and spot opportunities to improve DevOps practices.
Just as important as surfacing the right data is how engineering leaders interpret that data and use it to make decisions. That’s why analyzing data in context with other metrics is crucial to gaining a holistic picture of the software development pipeline. Then, rather than optimizing for a single metric, an ideal approach looks to balance multiple aspects of delivery.
One way to ensure balance is by looking at complementary metrics side by side. Certain combinations of metrics can help you identify where to make improvements or scale effective processes so your team can ship value to customers frequently, without sacrificing quality or stability.
Here’s what different metrics combinations can tell you about balancing speed and stability, and improving team efficiency.
Quick delivery
To get a sense of team efficiency and output, you can look at Pull Request (PR) Throughput alongside Cycle Time. This data can be mapped back to teams to illustrate which teams are shipping the most code, fastest.
Cycle Time, which is the time between when a commit is first authored and a PR is merged, acts as the overall speedometer for software delivery. A high PR Throughput (a total count of merged Pull Requests over a period of time) is a proxy for value being delivered to customers, and a low Cycle Time means value is being delivered often.
If Cycle Time is high, you can start by looking at PR Size. This is often directly correlated with Cycle Time – larger PRs take longer to move through the pipeline, and can lead to an increase in Cycle Time. If PR Size is an issue, coaching the team to adopt healthier coding practices, like breaking down work into smaller units, can be an impactful place to start.
Innovation
You can look at code maintenance metrics to get a feel for if teams are bogged down with technical debt, or if they’re achieving continuous innovation.
Viewing Rework alongside PRs Merged, for example, can illustrate the direct relationship between refactoring and innovation. Managers can also look at Cycle Time alongside Maintenance Rate. A long Cycle Time and high Maintenance Rate could indicate that teams are being slowed down by working with legacy code.
The Innovation Rate metric refers to changes that are made up of brand new code. When compared to Maintenance Rate – changes made to code older than three weeks – managers can see whether maintenance work is taking away valuable time from building new features.
Managing up
Certain metrics pairings can reveal insights that are valuable to share with executive stakeholders. If you want to show the impact of a strategic decision, like hiring more engineers, consider pairing PR Throughput with Active Contributors. If new team members are up to speed and contributing like seasoned team members, you’ll be able to clearly demonstrate how the team’s output has increased with new contributors.
Actionable DORA metrics pairings
The DORA metrics are four key engineering metrics identified by the DevOps Research and Assessment Group as the most strongly correlated with success for an engineering organization. Pairing these metrics with other engineering data allows leaders to drill down and understand what’s blocking engineers or contributing to high performance.
Deployment Frequency (DF) and PR Size
Use PR Size as a starting point for investigating a low Deployment Frequency. If a correlation is found, coach team members to break work into smaller PRs to see if DF improves.
Mean Time to Recovery (MTTR) and Revert Rate
A long Mean Time to Recovery could indicate a high Revert Rate, therefore disrupting production and lengthening the time it takes to recover from an unplanned outage or defect. If you notice a correlation, you can drill down into each revert and see whether the issue is a defect or an undesirable change.
Change Failure Rate (CFR) and Unreviewed PRs
If a high CFR correlates to a high percentage of unreviewed PRs, consider adjusting processes to prevent unreviewed PRs from being merged and causing issues.
Mean Lead Time for Changes (MLTC) and Cycle Time
If MLTC is low, it could indicate that your Cycle Time is too high. View these metrics in tandem with Velocity in order to check your assumptions, then dig deeper into Cycle Time-related metrics like Time to Open, Time to Merge, and Time to First Review.
Final thoughts
Context is key when it comes to engineering data. In tandem with the essential step of speaking to your team and gathering qualitative information, looking at metrics together can paint a fuller picture of your organization’s DevOps practices, provide actionable steps towards optimization, and ensure balance for delivery outcomes.
When coupled with your leadership expertise, a Software Engineering Intelligence platform can help you drive change in your organization.
