Applying a zero bug policy made it easier to prioritize bugs and increased team visibility and responsiveness towards bugs. As it’s a radical change, you will need to adapt it to your context regarding decision-making and time to fix a bug.
Simone Colosimo, senior QA manager at Dashlane, presented how they applied a zero bug policy at Agile Testing Days 2021.
The goal that they aimed for with their bug treatment strategy was:
We were observing a drop in the quality perceived internally and externally, and we considered that the way we were handling the bugs wasn’t sustainable. We decided to make a change of paradigm, introducing a different way to handle bugs to put quality and the customer back at the center of our product and to take back control of our bug backlog.
Using a zero bug policy, the classification of a bug is binary: it’s either a bug or it isn’t. As Colosimo stated: "If you can live with it, it’s not a bug, it’s an improvement." According to the policy, if the issue is considered a bug, it has higher priority compared to upcoming future developments or improvements.
Colosimo mentioned that the zero bug policy approach would be too rigid in their context, especially regarding the prioritization and the time to fix a bug, causing considerable pushback from teams, as it was a brutal change compared to their previous practice. They adapted it around decision-making and bug handling:
Whenever a new bug is opened, the product owner of the team decides whether the bug needs to be fixed, or if it should be closed.
Any new bug which is confirmed must be fixed within 30 days after the bug has been opened. We accept that some bugs could be fixed after 30 days, but this should be an exception and not the rule.
Using the zero bug policy has brought them several results:
Easier prioritization:
- We are now dealing with fewer items, which makes prioritization easier and more effective. This ensures that we are working on the most important things while limiting the risk of missing critical issues.
Improved team visibility and responsiveness towards bugs:
- Having a pile of bugs in the backlog is demoralizing for the teams, as it can desensitize teams to fix bugs; a new bug becomes just another bug.
- By dealing with bugs as they happen, teams developed a new attitude towards bugs and a greater sense of pride about quality, because their effort has an immediate consequence on the product’s quality.
InfoQ interviewed Simone Colosimo about applying a zero bug policy.
InfoQ: How do companies typically manage bugs in their backlog? What are the drawbacks of this approach?
Simone Colosimo: When quality is not top of mind and you have teams rushing to deliver new features, more often than not, non-priority bugs will be deferred and start piling in your bug backlog. It is rare in those situations that you ever take the time to fix those bugs.
The typical way to reduce the amount of these bugs is to do a "spring cleaning", aka dedicated sessions led by QA for bugs reproduction to close them if outdated or already fixed or to prioritize them if still valid.
This approach has two main consequences: we attribute a low value to the bugs reported, as a new bug is just another bug, and over the years the backlog will increase, making its cleaning more laborious and painful.
InfoQ: Why did you adapt the zero bug policy at Dashlane?
Colosimo: The motto we use is "be honest, be accountable."
We encourage an honest attitude against bugs: if the bug is important to fix, let’s do it. Otherwise, instead of deferring it, it’s better to close it. When a decision is required on whether a bug will be fixed or not, the accountability of decision makers will increase.
We needed to close the loop. The product owner (decision maker) should share their decision with the bug’s original reporter and also with the team. They should take time to explain their decision and justify the logic behind the decision.
From the reporter’s standpoint, it’s much clearer: they will receive either an explanation of why the issue won’t be fixed, or the issue will actually get fixed in a reasonable amount of time.
InfoQ: What results and benefits did you get?
Colosimo: Next to the easier prioritization and improved team visibility and responsiveness mentioned earlier, we had some additional benefits:
Increased reporter satisfaction:
- We’re more reactive and more communicative.
Reduced technical debt:
- We no longer wait several months before fixing a bug, which forces stakeholders to put themselves back in the context, increasing the risk of breaking other areas as the code has changed in the meantime.
- The size of our backlog has decreased by 80%.
- On average, we fix a bug within a Sprint (14 calendar days). This is not perfect yet. We still have some bugs that occasionally breach our 30 day policy, but we work hard with the team to meet that commitment to quality.
These are positive and promising results, and we’re still striving to do more and reach new achievements.
InfoQ: What have you learned?
Colosimo: The internal discussion around the proposal for a new bug treatment strategy was passionate, intense, and instructive. I learned that it’s key to involve the stakeholders, within and beyond engineering, in the development and finalization of the proposal. While we didn’t incorporate all of the feedback, honest communication has been fundamental for the next steps. Also, the involvement and support of the executive team was an important aspect: if the execs don’t support the change, it will be very complicated to implement it.
Last but not least, I learned that empowering key team members was the secret to a successful implementation. They must be autonomous, we must provide them with the right tools and train them to apply and "live" the strategy in their daily work.
InfoQ: What’s your advice to companies that want to adopt a zero bug policy?
Colosimo: Don’t be afraid of adopting a radical change in your process. If you notice bugs piling up in your backlog but you see few actions being taken to regain control over it, then you know it’s time to act.
At the same time, you must do the context assessment attentively. For example, suppose you try to apply a policy as-it-is. In that case, you could risk missing the target and producing a cobra effect; a backfire leading to an undesirable result that is contrary to the initial intentions.
In our case, we amended the zero-bug policy because we considered that changes would have a bigger, more durable impact and limit pushbacks. We assigned the decision-maker role to the product owner. Maybe in your case, it will be the tech lead, or the QA in charge of the project, and maybe you won’t need to amend the policy as we did. Context is king.
