While, zero defects sounds very good to hear, is it really possible or is it an unachievable goal? Many organizations adopt a 'zero defects methodology'. Does it really mean anything?
According to Jim Bird, the cost of 100% perfection is extremely high. Once teams reach the optimal level of 90% defect removal, the returns from further removal of defects is significantly lower for the unjust cost associated with it. Jim, quoted Ken Beck and Martin Fowler's book 'Planning Extreme Programming' in which they mentioned,
For most software, however, we don’t actually want zero bugs. Any defect, once it is in there, takes time and effort to remove. That time and effort will take away from effort spent putting in features. So you have to decide what to do.
Likewise, Michael Dubakov suggested that having a Zero defects mentality might create more problems than the benefits that it can bring. According to Michael, the unpleasant effects include
- Not enough courage to refactor complex, messy, buggy, but important piece of code.
- Can’t make important decision, instead make less risky, but wrong decision.
- Do everything to avoid responsibility, that leads to coward and stupid behavior.
Michael suggested that in reality having bugs in a production system is normal. This does not mean that teams should get complacent and avoid fixing bugs. However, this does imply that the so called 'last bug' is a mirage.
Jim suggested that one should be selective about the bugs that need to be fixed. Teams should first determine the importance of the bug to business operations by validating its severity and frequency. As a next step, technical factors like 'cost to fix' and 'risk to rest of the system' need to be considered before going ahead with the fix.
Zero bug tolerance naively assumes that it is always good, it’s always right, to fix a bug. But fixing a bug is not always the right thing to do, because with any fix you run the risk of introducing new problems
Joel Spolsky suggested that Zero defects does not mean zero in the literal sense. It means that at any given time, the highest priority is to eliminate bugs before writing any new code.
So what are the best ways to minimize defects?
Mark Windholtz, placed significant importance on TDD when he suggested,
Test-First coding that is fundamental in achieving the goal of Zero Defects. Test-First coding requires that an automated unit-test is written before the production code and the test-to-code cycle time be 5-15 minutes long.
Likewise, Michael Dubakov suggested a combination of TDD, continuous integration, automated regression suite, root cause analysis and high development skills to reduce the number of defects.
Rolf Gotz mentioned a list of 10 principles for getting to zero defect systems. Some of them included
- Customers and software developers who like each other.
- Small and easy scope of your requirements, increasing in small steps.
- High-value requirements are the initial focus.
- Acceptance criteria are paramount.
- Problems first (then the requirements).
- Performance requirements first.
Thus, though the system should have minimal defects, striving for zero defects is an unending goal. The key lies in knowing when to stop. As Jim advised,
Knowing when to stop fixing bugs, when you’ve reached the point of diminishing returns, when you should focus on more important work, isn’t easy. Knowing which bugs to fix and which ones not too, or which ones you can’t or shouldn’t fix now, isn’t easy. And you will be wrong sometimes.