Zero Bug Policy: The Myths And The Reality

What Zero Bug Policy Is Not

You may have heard the phrase “zero bug policy” sometime in your testing career! A sound zero bug policy doesn’t mean the application has no bugs. It’s about focusing on building a quality product for the user.

You may ask: isn’t raising bugs what testers are supposed to do? Yes, and problems will still be found and should be brought to the team’s attention as soon as possible. But they’ll be named something different and will likely be handled quicker and more effectively. 

Zero Bug Policy, Simply Stated

As a reminder: let’s define “bug” simply so we can explain zero bug policy:

“A software bug is a problem causing a program to crash or produce invalid output. The problem is caused by insufficient or erroneous logic. A bug can be an error, mistake, defect or fault, which may cause failure or deviation from expected results.” https://www.techopedia.com/definition/24864/software-bug-

The premise of zero bug policy is to redefine what is classified as a bug, Peter Hilton (https://hilton.org.uk/blog/zero-bug-policy) explains that the aim of zero bug policy is to fix bugs before adding any new code. Switching gears frequently to fix bugs and test the new code can lead to unpredictable development timelines, which a zero bug policy can help to address. 

Within my company, bugs are called “live issues” and are immediately documented as user stories. This means that they get prioritised alongside all other user stories, so they carry the same value (in theory). 

So bugs are categorised as follows:

• Bug found in production -> “Live issue” created as user story
• Bug found during feature testing 
  • Must be fixed before release -> User story, prioritised for development and testing
  • Feature will be released with bug present -> User story for that feature is updated to account for bug
• Bug found during regression 
  • Must be fixed -> User story created and added to backlog
  • Satisfied with existing workaround -> No new user story created

Bugs found during regression testing can point to a gap in testing: for example, tests were not updated when some requirements were changed. These too are categorised as user stories and can be prioritised accordingly.

Case Study: Zero Bug Policy In Practice

To begin with, the difference between live issues and bugs was confusing and required explanation and discussion. Questions like “aren’t they the same” and “live issues should be dealt with apart from user stories” arose right away. The answer to both questions was, of course, “yes,” which led to us agreeing to disagree. We would call bugs “live issues” if they existed in production. Also, we created a fast track stream for urgent live issues: these would bypass the normal process of being added to the backlog and would appear directly in the current sprint.

Secondly, the idea of not raising a bug that was found during regression testing was difficult to comprehend. Again we made a compromise by classifying these as bugs in the backlog. Like anything about working in Agile, it’s fine to make compromises and adapt the policy to the way your team wants to work. The policy provides guidelines, but the team wanted to distinguish issues found by customers from those found by internal team members.

During the transition period, there was some confusion about where user stories should live on the scrum board. For example, a user story would remain in the “In Test” column of our Jira board, when actually the requirements hadn’t been met and so should have been put in the “In Development” column.

The product owner was unsure where to look for issues related to the user story, as these were only comments now. Sometimes there weren’t even comments, because the change was the product of an in-person conversation between a developer and a tester. The occasional absence of documentary evidence was uncomfortable for some at the start. 

This is a hallmark of a zero bug policy: sometimes the issue was very small, like a typo or a one-line change. However, for more complex issues, a discussion between the tester, product owner, and developer would occur. And they would create a new user story to avoid introducing undue complexity into the existing user story. This conversation would happen in the form of a demo, with responsibility for creation of a new user story eventually handed over to the product owner.

What Are The Benefits Of Zero Bug Policy?

My team noticed some major benefits after adopting this approach:

• Testers reinforced their relationships with developers by having conversations instead of communicating through tickets
• Team focused on getting user stories over the line instead of concentrating on the bugs raised
• Product owners gained more control over what was important to fix right away versus what was actually a “nice to have”

Don’t get me wrong: it does take some convincing to get the team on board. Also, it does take some time to adapt to the new ways of working. But once you get over the hump, the new way of working starts to make sense and improve quality both of product and communication. 

To Wrap Up…

Zero bug policy isn’t about achieving bug-free software. The focus is on what’s important: shipping quality software to the users. And it removes the burden of reporting bugs and signing off on user stories from testers’ shoulders, relieving them of the disliked “gatekeeper” role. 

The following process changes will probably be needed for your team:

1. Pushing user stories back to development instead of raising separate bug tickets
2. Releasing working software which is “good enough”. Problems in released software can be added as user stories to the backlog and prioritised accordingly

Why not give it a try? But I recommend dispelling the “zero bug” myth first.

For Further Information

• How Product Owners With Testing Experience Can Promote Quality, Mirza Sisic
• When Testers Deal With Process Debt: Ideas to Manage It And Get Back To Testing Faster, Melissa Eaden