A recent discussion thread on the Scrum Development Yahoo Group examined the value of process checklist tests such as the Nokia Test or the Joel Test. Some see these tests as the starting point for a rich agile maturity model, others worry that this could lead to prescriptive approaches to agile, which would miss the whole point of inspect-and-adapt entirely.
Bas Vodde is generally credited with creating what Jeff Sutherland later named "The Nokia Test." Bas described the history this way:
The "you are not agile when" slide was created about four years ago within Nokia Networks (now Nokia Siemens Networks). The purpose was, for the Agile coaches (!!) to quickly evaluate whether a product is actually serious in its move to Scrum. It was the minimum bar, if a product is not even passing these points (we do Agile, our iterations are 2 months long) then it might not be worth to put effort in coaching them (yet).
...
The simple list ... ended up in a presentation I gave about Nokia Networks, which made it public. Jeff found it a useful simple list, took it, and called it "the Nokia test" (it was simply called "you are not doing iterative development when...").
Overtime it got "improved" into something different, but the name always kept the same.
Peter Stevens shared his experience tweaking the Joel Test for use in his own practice.
My goal on modernizing the Joel test was to get a tool to help me as I start coaching a project which is struggling to be agile. But I discovered quickly that the questions are not that helpful. The real problems become obvious very quickly as I watched the team do its sprint retrospective and sprint planning. Reacting to what I see is more important than doing an academic evaluation.
Tathagat Varma suggested that instead of asking "Are we doing these practices?" that organizations should examine the results of their practices in areas such as: employee motivation, team productivity, and customer satisfaction. At this point Ron Jeffries joined the discussion, pointing out that examining results won't tell you if a give approach is agile.
I mean that not all means of going after results are compatible with Agile values. The Agile values and principles can be found in the manifesto. Anything substantially different from those might be good, but it would not be properly called Agile.
All of this begs the question, "What is the point of the test?", or more specifically, "What do you hope to learn from applying it?" Bas, the originator of the "Nokia Test" shared this these thoughts.
Related to the original questions, are these tests useful? It depends very much on what you use them for and to make sure that the people who use it actually understands what it means (as it was only 5 questions...). In general, I do not find "agile maturity model", "Agile assessment models" or "agile tests" useful since they tend to be a simplified view on a complex subject.
It would seem that the 'Nokia Test' served its original purpose, to help the agile coaches at Nokia determine which teams might be ready to benefit from coaching. Can it be adapted to other purposes? While the answer is almost certainly yes, many caution against applying it without carefully examining, and possibly adapting it first. Jim Brosseau had these insight's back in December of 2007:
First off, I think that if you are not Nokia and you have decided to use this test as your golden standard, you have already missed the point about agility. No single defined approach will ever be optimal for all your projects in your organization.
Jim then went on to disassemble the Nokia test, separating general concepts, which might be more valuable, from situation-specific details which might not be appropriate in other environments.
In the end, what can be learned from all of this discussion? Simple checklists can provide a useful quick-and-dirty assessment. They won't tell you if the practices are actually delivering results. In the end, a checklist is no substitute for an understanding of the agile principals and practices along with insight into how they might be effectively applied in a given situation. This, combined with a culture of inspect-and-adapt (continuous improvement) is likely the real key to a high performance agile implementation.
