Great reporting can help engineering teams boost alignment, progress towards goals, and improve their strategic planning.
To do that effectively, you’ll need to leverage data. Many engineering organizations stumble when it comes to introducing data. Some fall prey to common pitfalls – assuming numbers tell the whole story, measuring the wrong things – while others are so afraid of making a misstep that they avoid it altogether.
If you’re considering introducing data in your organization, there are a few best practices that will help you use data responsibly, constructively, and effectively.
1. Embrace transparency
Transparency is a critical characteristic of successful data-driven organizations. Open communication can help ensure everyone feels comfortable about what data is being looked at and how it will be used, while reinforcing key principles for using data appropriately.
Before you roll out data in your organization, make sure everyone understands:
- What data is being looked at:
‘Data’ is broad, so it can help to get into specifics. Are you aggregating data at the team level, or digging into individual metrics? Are you looking at big-picture metrics like pull request cycle time, or getting granular and assessing every step of the software development lifecycle? Make sure your team knows what data you plan to analyze. - Why this data is useful:
Objective data can enhance visibility and boost alignment. It can improve goal-setting, ground conversations in fact, and help combat biases. It can empower developers to remove blockers and actively improve their team’s processes. There’s a lot you can accomplish by leveraging data. It’s important that your team understands exactly why you’re bringing data into the organization, and what you hope to accomplish. - Who will have access to what data:
The answer to this question differs depending on the organization, but in general, executives will be looking at different data than managers or contributors. In many organizations, individual contributors don’t have access to engineering data at all, or can only look at team-level data. - How the data will be used:
When engineering organizations start using data for the first time, the ‘how’ is often the biggest source of fear. Developers may worry that their data will be used against them – that they will be held to certain numbers, and penalized if they don’t meet them. When everyone understands how data will and will not be used (more on that below), it can help allay many of these concerns.
2. Use data to enhance conversations, not replace them
Even the most comprehensive data can’t tell you everything you need to know about your team and how it’s working. It’s important to place all quantitative data in context by gathering additional qualitative information from your team.
For example, you may notice a recent decrease in your team’s PR throughput, but you’ll need to bring that information to a standup, 1:1, or team meeting to determine exactly why. If you use that data to start a conversation with your team, you may learn that meetings are eating into your team’s time to code, or that unclear technical specifications are making it difficult to get work ready for production. With that knowledge, you’ll be able to work with your team on finding a way to remove their blockers and bring throughput back up to its usual level.
3. Interpret data as insights about the work, not the person
Engineering is highly collaborative, and any data you’re looking at will be impacted by multiple factors – no one person should feel personally responsible for a given metric. If you keep your focus on the work, rather than the individual behind it, you’ll be able to sidestep unhelpful conversations about blame and have more productive discussions about how to make improvements going forward. You’ll also be more likely to preserve your team’s psychological safety, which is critical to their ability to deliver.
Metrics should never be used as justification to punish or penalize a team member. Instead, missed targets should be seen as coaching opportunities, with metrics helping to keep conversations grounded in facts.
4. Focus on goals first, data second
Quantitative measures can be a powerful tool in helping teams meet their goals, but it’s important to make sure the data you’re looking at aligns with your goals in the first place. Spend some time considering your team’s priorities before deciding what you’d like to measure. Not all metrics are useful for every goal, and measuring the wrong thing – or too many things – can actually hinder your team’s progress.
Like any tool, engineering data must be used responsibly to have a positive impact. Yet with a little bit of thought and a commitment to key best practices, it’s possible to leverage data to drive progress and level up your team.
