Optimize sprint points to get nowhere fast

Monday, June 12, 2023

As developers, we can be metric obsessed. We tend to like objective measures of things. 99th percentile request times, CPU percentage, disk utilization. Nothing escapes our attempts to quantify it, not even our productivity: enter story points1.

We measure our productivity in some way by how much we get done. This is the quantity of work or complexity that a team can get done in a sprint. And once we have a metric, we ruthlessly optimize it.

We want to move fast, so we see how we can improve sprint points. What processes can we optimize? Can we get designs earlier, and plan things out a little better? Can we streamline and remove meetings?

We push story points up and up and up. Eventually they're at a new level, and that becomes the new baseline we have to hit. The urge to get it higher is there, and it's a ratchet that doesn't let the level slip back down.

But where are we going? That's sometimes delegated to product. Product worries about what we build, and engineering worries about how we build it. In the ideal world, anyway. But, here's the rub. We are all on the same team together. We are all going the same place. Code doesn't matter if it isn't useful, and ideas and product direction don't matter if they don't get implemented.

We're one team, and we should have the same direction. If we optimize for speed of engineering, we are sacrificing something else.

The problem is with our frame of reference. If we are zoomed in to what we get done each sprint, we are looking just relative to engineering and just relative to where we are. Are we moving? How fast? But we're not asking about where we're going.

If we zoom out and we look in terms of the destination, we get to the measurement that really matters. The ultimate metric that we care about is: how quickly do we get to the final destination of features that work for the users? To really stretch the metaphor, we usually measure the speed of our car, but we don't think about which direction it's pointed in. If we find a highway without a speed limit, we might get on that even if it can't take us where we need to go!

So why don't we measure progress toward our destination? Well, because we don't know where that is until we get there. If we knew ahead of time where we're going, then we could just measure sprint points since we would know what product direction is the most important one. But ultimately, we don't know that.

We know we got to a good destination once we get there. While we're on the way, we don't know what works and what doesn't.

So, what do we do instead?

First, don't throw the baby out with the bathwater. Sprint points are important. (Well, some estimation of productivity is important; relative velocity, as it were). We want to keep that measure, but we have to work to not optimize for it alone. It isn't the end goal, but it's a useful diagnostic signal. If you can't get your car above 20 MPH, you want to go get it checked out, but that doesn't mean you always want to floor it.

And so we can look at other metrics. These are going to be things that center around exploring the landscape so that we can figure out the direction to go in more effectively. Some candidates that come to mind:

  • Time to ship an MVP of a feature: the shorter you make this, the faster you can get feedback and determine whether or not it's the right direction
  • Time to get user feedback on a new feature: again, shorter gets you feedback faster
  • Time to complete an iteration on a feature: the more iterations you can fit in, the more times you can get feedback, and the more you can course correct
  • Amount of user feedback you can get per timeframe: this will help you know where you're going

It doesn't really matter what the specific metric is, as long as you switch from optimizing for productivity alone, and include consideration for the ability to explore and get feedback. I don't think these metrics are north stars that should be optimized for independently, either. All metrics in moderation, as they say.

This isn't something engineering can do alone. This isn't something product can do alone! Making great software is a team sport and is highly, intrinsically, collaborative. Working together to measure the right thing and shift focus to the final destination is one of the keys to making great software and great products.

Let's not forget that where we get to matters a lot more than how we get there2.


1

Or your estimation technique of choice. Personally, I prefer wall clock time, how long something will actually take. This is controversial, and is a subject for another post.

2

In the sense of process, not in the sense of "ends justify the means." It's not okay to do unethical things for a just end, but it is okay to change processes to get to a better end outcome.


If this post was enjoyable or useful for you, please share it! If you have comments, questions, or feedback, you can email my personal email. To get new posts and support my work, subscribe to the newsletter. There is also an RSS feed.

Want to become a better programmer? Join the Recurse Center!