The Agile community has had a number of attempts over the years to determine the best way to measure the effectiveness of an organisational Agile adoption, with some of the better known attempts including the Agile Scaling Model, the Nokia Test, the ThoughtWorks Agile Assessment and Sidky Agile Measurement Index (SAMI). The measurement debate has reignited again recently in the Agile blog world.
Esther Derby in a recent post entitled Metrics for Agile recalled the type of metrics people usually ask for when they wish to gauge how far along they are in their Agile adoption:
- Number of teams using Agile
- Number of people trained in Agile
- Number of projects using Agile
- Number of certified coaches.
She suggests that these metrics are likely to lead you astray and correct metrics are those that demonstrate that you are trending towards what you set out to achieve through an Agile adoption. Her three suggestions are:
- The ratio of fixing work to feature work
- Cycle time
- Number of defects escaping to production
Her final advice is:
When you think about measures, be wary of target numbers. Measuring against targets almost always causes distortion. That means that people will behave so as to reach the target, perhaps in ways that are counter to the actual goal behind the target.
A recent series on the Rally Software Agile Blog by Isaac Montgomery entitled Measuring the Impact of your Agile Investments had some similar insights:
The Agile community’s typical response to [how do we measure the success of Agile question] has been some form of an Agile maturity assessment – such as the Nokia Test. These tools are clear, easy to use and can be extremely effective in helping organizations assess their adherence to good Agile practices. Yet, when used in isolation, they can leave senior leaders unfulfilled – and miss an important opportunity for aligning Agile to fundamental objectives.
He suggests the question should be:
…if Agile Success = Business Success, then the real question is: “How do we measure business success?”
Isaac suggests that metrics should fall into the following categories:
- Productivity
- Quality
- Responsiveness
- Customer Satisfaction
- Employee Satisfaction
- Predictability
Finally, Gil Zilberfeld has some simple advice regarding the Truth about Agile Adoption when he contrasts Agile adoption metrics against the often cited Standish Group CHAOS report:
Get your own numbers. Measure your own goals and define your success. Letting someone else do it for you (or not doing this at all, for that matter) is just plain dumb.
How do you measure your organisations Agile maturity and what metrics do you find to be the most useful?