Story points have their place in Agile, but they leave much to be desired for engineering leadership.
Engineering teams will continue to use story points for the foreseeable future, and this post isn’t an attempt to try to sway your teams from them. To be clear, I won’t be picking a fight with story points in this article; I’ll be acknowledging their limitations and explaining why they shouldn’t be used as the foundation for measuring engineering work’s strategic alignment to the business.
Story points are a necessary tool for Agile sprint planning
If there were a better alternative to story points, even for planning purposes, I imagine that teams would be rushing to try it. But they’ve been a mainstay of Agile for a reason.
Sprint-planning teams (project management and engineering) need a way to measure all of the various factors that could impact the total work for a given task. Story points are a best attempt at assigning one value to represent the amount of work and risk involved for each task. And currently, story points are the most widely accepted way to communicate with project management about the effort that backlogged feature work will take.
For these two reasons, story points have enabled the sprint-planning model, and we should expect them to continue to be used until technology or methodology improvements enable something better.
Story points don’t work for strategic alignment and retrospection
Story points work for trying to plan and assign tasks toward completion of a project on a given team, but their value breaks down for representing engineering work across multiple teams, and especially for planning across functional organizations.
Consider the scenarios when you as a leader would want to collect information regarding how your engineers have been spending their time. There are two primary reasons: to inform your decision-making and to report to your business peers.
For strategic decision-making
For strategic decision-making, consider the types of insights that would help you. Productivity, efficiency, and allocation (what your teams are working on) are the types of metrics engineering leaders frequently cite. For any of those insights, what would a retrospective on the assigned story point value on the feature work actually provide?
Are you confident all of your teams are assigning story points in the same way? For most teams, the reality is that their teams assigned story point values differently. Even if you could turn that effort estimation into an actual final value, and your teams follow the same criteria for assigning story point values, you would still need to translate story points into time to properly calculate recommended productivity, efficiency, or allocation metrics. That’s a hard no for an Agile advocate, so let’s just not use story points from the start for this specific purpose.
For reporting to the business
Story points won’t translate to any other area of the business. And as soon as we become heads of engineering, we’re responsible for translating engineering work into the language of the business. As a general rule, if we have to explain any metric to a business peer, we should probably try to find another metric first.
Additionally, story points don’t translate easily to costs. When trying to evaluate where to invest engineering resources (likely a conversation with product, and business leaders), understanding the costs of the products and features you’ve already built is extremely valuable. When reporting with story points, it’s nearly impossible to parse out the factors with costs associated with them from those that don’t (for example, risk vs number of engineering hours on a task). Leaders require a better unit of measurement for these purposes.
Measuring engineering effort in FTEs
For strategic planning, leaders can represent the amount of engineering work on a particular feature or initiative in terms of time. A common way of representing that on engineering teams is by representing development work in full-time engineers or FTEs.
Yes, doing this manually can be painful for everyone involved. There are exceptional organizations that have overhauled their engineering operations to do this, but they have the battle scars to show for it. For the rest of us, analytics tools like Engineering Management Platforms provide a much easier route.
Reflections
Story points have a purpose, but leaders shouldn’t use them in all instances just because that’s what they can measure today. Elevating the practice of engineering management requires looking to the future of what should be measured. Engineering orgs should be investing in being data-driven and developing their work allocation models. The future of engineering management requires it.
