Measure Teams, Not Individuals

In response to requests from users of his company's agile project management and life-cycle product, Target Process, Michael Dubakov poses his warnings against the measurement of individual velocity or estimate accuracy on an agile project. His view is that measurement of these metrics will provide no more useful information than is already available with their team-level equivalents and may also encourage teams into behaviors that reduce effectiveness.

In a late 2007 post, Dubakov raised the issue of agile teams' desire to measure individual velocity. Using an example of two developers, Ted and Jerry, Dubakov illustrated his view that a set of historical 'individual velocity' measurements will provide a team no better mechanism to plan future iterations than does the team's overall velocity measurement:

If Ted completed several tasks with 40 hrs of effort in total and Jerry completed tasks with 25 hrs effort only, we should say that Ted has better velocity in last iteration. Does it mean that Ted is a better/faster developer? Not at all. There are hundreds of reasons why Jerry completed less ... OK, but what about average velocity? Surprisingly, Jerry’s average velocity is 54 hrs per iteration. Gosh! What happened with Jerry last two weeks? Will his average velocity help us to make correct iteration plan? If we sum up individual velocities all developers will it help us to create a better iteration plan? No, since we already have Iteration Velocity metric and it will be exactly the same.

More to his point, he continued by outlining two ways this metric is likely also to work against the ideal operational goals of an agile team:

1. a harmful focus on individual performance, over team performance, which he notes is likely to discourage team members taking time to help one another out
2. an increased tendency towards individual work assignment, over team commitment

Spurred by Michael's point and a related thread, James Carr soon added this reminder about the use of velocity in general:

Velocity is not about [assessing] performance… it’s about providing your customer with a close to accurate amount of “points” they can spend on features this iteration. Remember that.

In a more recent post Dubakov revisited the subject, this time adding a warning against the measurement of individual estimate accuracy. He first pointed out that this metric isn't even feasibly useful unless, first, the estimates were given by individuals and, second, the team tracks time spent for all tasks. The major problem with these conditions, many in the agile community would assert, is that both are likely to work against fundamental agile principles fostering teamwork and simplicity.

To illustrate the misleading effect which measurement of individual estimate accuracy may have, Dubakov goes back to the hypothetical developer Ted:

We can calculate all Ted’s assignments and spent time and define estimate accuracy for next iteration, let’s say it is 0.7.

OK, but how we can use this metric? Well, if Ted subscribed to 60 hrs work it means he will spend 85 hours and for 2 weeks iteration it means at least 5 hrs overtime. Ted should take this information into consideration and remove some tasks from his ToDo. This works if Ted estimate accuracy remains the same, but is that always the case? In reality Ted’s accuracy may vary from 0.5 to 0.9 and exactly for next iteration it may be 0.9 and in this case Ted will be able to do all committed work.

InfoQ's Deborah Hartmann takes Michael's point a step further, questioning the measurement of any time-based estimate accuracy at all, individual or team:

To calculate this kind of estimate accuracy, teams would have to expend effort to capture detailed "actual" hours, a practice not advocated by any Agile approach I've seen. The classic "units of work planned" vs "units of work fully completed" measures estimate accuracy in units of more value to the customer: work delivered (story points, ideal hours, bananas, whatever).

The tracking of estimate accuracy in terms of actual hours expended not only adds no more information to the team, it also imposes a new form of waste! I agree with Dubakov, Carr and others: I don't see how this could be a helpful metric for most teams, and I'm happy to see that it was quickly removed from TargetProcess once the point was raised. This is the kind of responsiveness to change that we expect Agile teams to exhibit.

Dubakov, Carr, and Hartmann all agree, with regard to both individual velocity and individual estimate accuracy, that measurement of these metrics not only provides no more useful information than is already available with their team-level equivalents, but may also have a tendency to encourage teams into behaviors that work against true agility.