How many bugs do you manage? 100, 200, 2000? Perhaps you don't know what the exact number is, because it keeps changing.
I know exactly how many bugs we have – zero.
How much time do you spend in bug triage meetings and on bug management, moving them around from release to release?
As a group manager I spend half an hour once a month. And I never see the same bug more than once.
Did you ever wonder how to handle bugs when working in Scrum or other agile methodologies?
The answer is the 0 bugs policy. It is based on my experience in several Scrum teams in different products over the last 5 years.
Other ways we have tried have failed: for example, we tried prioritizing the bugs as part of the product backlog, and it didn’t work. The product owner always preferred to have new features instead of fixing existing bugs, so all open bugs were always deferred to the next iteration.
With the 0 bugs policy, it just works.
This article describes what the 0 bugs policy is, and why in the author's opinion it is the only way to go.
What is the 0 bugs policy?
The 0 bugs policy is an advanced yet very simple process for handling bugs.
It underlines a simple rule you need to apply: whenever you encounter a new bug, you should either fix that bug, or close it as "won't fix" and don't think about it again.
How does it work?
Bugs can be generalized into 2 categories:
- Bugs that were opened during the development of a new feature.
If you are working in Scrum (or any other agile iterative methodology), these are the "in sprint" bugs that were created for the new user story you are currently implementing.
These kinds of bugs must be fixed right away, otherwise the story/feature is not really DONE, and you are violating a basic agile rule saying that: DONE is DONE. The story/feature can only be considered as completed after it was fully tested and approved by the product owner.
- All the other bugs (non-sprint bugs) which can be broken down into:
- Regression bugs – bugs that were created and found in feature A due to the development of feature B.
- Customer bugs – bugs that were reported by customers or any users of the product who are not part of the development group.
- Bugs that were found after the development of the story/feature was completed. This should not happen, but sometimes does: for example, when the testing coverage was lacking, when doing bug hunts, or hardening.
So what can be done with bugs from the second category?
- You can either fix them right away (or in the next sprint, but not later).
- You can close them as "won't fix" if the value of having the bug fixed is less than the effort of fixing it.
- You can defer the bug to the future (several month from now or future releases).
Never defer the bug. If you don't fix it right now, chances are that you never will.
So, what should you do? Just fix it, or just close it. That is the simple rule of the 0 bugs policy.
An example of a bug that must be fixed is:
2 users cannot enter the UI simultaneously …
An example of a bug that you can decide not to fix if the effort is more than 2 days work is:
When using Safari, with a screen resolution of 768x1280, the help text in the login page is blurry. For other browsers, or other resolutions there are no issues.
You might ask why can these bugs cannot be deferred, in order to fix them later?
The answer is that it will cost a lot more to fix it later. Several weeks from now:
- You will not remember nor will you understand this bug as well as you do now.
- The right environments – both for validation of the bug and for development a fix – might not be available. It means you will need to invest more time in setting up the environment.
- The code that caused the original bug might have changed and the behaviour will be slightly different.
But the fact that in the future it will cost you more to fix the bug is just a small part of the problem.
The second part is that you will need to maintain, triage and manage the deferred bugs. Whether they are managed in their own backlog (in some bug triaging system) or as part of the main feature backlog, you will need to prioritize them, and that takes time. It takes longer for old bugs, because you don’t remember nor understand them as well as new bugs. Triaging bugs can be annoying.
From my experience when you have several managers, some developers, the product owner, and maybe some architects as well, seating in a bug court (trying to decide what to do with a bug) the discussion turns into a war with no winners:
- Development manager: "It is too risky to fix this bug now, let’s defer it…"
- Product owner: "You don’t care about the user experience, you don’t care about the customer"
- Developer: "We can actually create a simple Go module instead of Java, in order to improve the performance on the target machine"
- Architect: "I suggest that we move from extJS to Angular; the capabilities are huge, but we need to remember that this fix might have a security impact as well"
- Quality Tester: "Testing that fix is easy: I can do it in 2 hours. Of course, if we need to test it in all browsers, and with multiple users, it might take a few more days; also we need to make sure there is no regression from scale perspective"
You will never fix this bug. It will be in the system forever. The number of bugs will slowly get larger. You will keep dragging them with you wherever you go. Triaging them, over and over again. Showing them in your weekly reports; calculating them in your quality index.
Why will they never be fixed?
- If you have decided not to fix it now, it means you have more important things to do, now and in the future releases.
- As time goes by, more and more bugs will be deferred, and then this bug will have to compete both against new features and new bugs. So, if the bug isn't fixed now, when there are no other important bugs as well, why do you think it would be fixed it later?
Why not fix non-sprint bugs during the hardening phase in the end of the release?
In the hardening time you should strive to stabilize your release. You should invest effort in finding the unknown unknowns, and not fixing small bugs you have decided to defer.
In any case, you can't fix all bugs, so some will be deferred to the next release. In the next release you will have new bugs; so in no time, you will have low priority bugs that you will never fix, but you spend time maintaining them.
Maybe after 4 years you decide to close them, since the overhead isn't worth it. Imagine how much time you would save if they were closed when the bug was created.
Why not fix non-sprint bugs during dead times?
You should not have dead times. If you do have it, you have bigger problems than handling low priority bugs.
Why not fix non-sprint bugs in the next release when we'll have more time?
Really? No features in the next release? No customers? No deadlines? No milestones?
And of course all the developers "love" those 2 weeks of bug fixing … having a few bugs in every sprint will make the team much happier.
If you are at the beginning of a new release and the features backlog is empty, invest time in innovation or social activity. You will benefit a lot more.
Why not fix non-sprint bugs in dedicated sprints for bugs squashing?
We have tried this approach before, and it has some benefits. The product owner doesn't have a choice, since the entire team is just fixing bugs; with everyone fixing bugs, it gives a sense of "we are all in this together" kind of spirit.
But, nevertheless, I do not recommend it, since this approach just minimizes the overall problems. It does not eliminate accumulating a large bucket of bugs; and you are still faced with the problem that we are fixing bugs late in the game, when it’s harder.
Why not fix non-sprint bugs as part of the backlog?
Actually, this is not a bad option. There are a couple of problems though:
- The product owner will have a hard time understanding the bugs and prioritizing them over new user stories: most likely she would tend towards the stories. Which is more important? Removing a redundant scroll bar in a pop up window or adding search functionality? This will cause low quality product and frustrated R&D.
- Each bug that you will not fix now will cost more to fix it later. It is just not cost effective to defer bugs.
Here's what happened when we have tried having a mixed backlog of bugs and stories. We tried the pre planning with several stories and bugs prioritized together:
During planning the team could only commit to the first 3 items
The product owner decided to push item number 3 down the priority list, since the "stop application" feature was critical for some proof of concept
The product owner was still not happy since the "search" story was not committed
We ended the planning with the team committed for the first 3 stories, and to do a stretch effort to fix bugs as well. The bugs were not fixed. In the sprint that followed a similar pattern happened again, but only there were more bugs at the backlog, and new features.
Now we have agreed that 0 bugs policy is the only way to go.
Why is it not more common?
There are several reasons why the 0 bugs policy may not be more common:
My product already has 1000 bugs in the backlog, how do I start?
Spend a week going over all the bugs. Either close the bugs, or fix them in the next sprint. You need to get to 0 bugs and then maintain it.
The fear factor
This is probably the actual reason why people would not like to pursue the 0 bugs policy. Several groups I have introduced this methodology to have declined to even try it, due to fear. This what I’ve heard:
If we close this bug, it will be gone! We will lose it forever!
Yes, that's right! Isn't that feeling liberating?
But, what if we close this bug and a customer sees it in production and will open it again?
Excellent, maybe now the bug will be considered more important and decide to fix it now!
We are 2 person startup? Do we really need this 0 bugs policy?
If you are 2 person startup you don’t need any policy: you should not really have bugs, and you should definitely not triage them.
Isn’t one of agile main practices to prefer people over processes, and having a "0 bugs policy" defies that practice?
Yes, it does, and from my experience I always prefer to solve problems by culture change, and not by adding processes; the 0 bugs policy is the one exception for this rule.
Final notes
I can honestly say that applying the 0 bugs policy is not trivial, but it is very cost effective. The hardest part for me was probably convincing the product management folks that this approach is the right and only way. It required them to release some control, and that is not an easy thing to do. The best thing about it is to go home knowing that you do not have this humpback slowing you down in the way.
Another side effect of the 0 bugs policy is that now our developers strive for higher quality; no developer likes to have bugs in their code. With the 0 bugs policy, they understand that if they don’t fix their bug now, it will just be closed. So the product and culture of the development organization moved towards higher quality standards.
This blog was influenced by Joel Spolsky's blog "Software Inventory", but mostly by my colleagues and partners in VMware Israel.
About the Author
Gal Zellermayer is a Jack of all trades, master of one. Managing different R&D groups in VMware for the last 5 years, Gal has vast experience in all aspects of the development life cycle: from innovation, through building the product backlog, leading the implementation while maintaining high quality and delivering the product to the customer. But his true passion and mastery is building the perfect software house. A team that can deliver any product with a high quality in a fast manner. Gal is passionate about new technologies, products, and also about the process, but the thing that excites him the most are the people he works with. Gal gets proud when people that he has mentored and coached get advance in their careers.
