Strategies to lower bug count

Question

Looking for ideas to ensure bugs count kept low say about 40-50% of total stories submitted to testing, here are the steps we tried

1. Before QA, I ask the story owner to demo the work
2. Code review, hoping to catch bugs before going to QA, even thought code reviews are for checking code compliance with our standards, unit tests etc..
3. Ensuring that devs understand the story and the acceptance criteria

We have a mandate from the management to lower our bug count, regardless of severities. Appreciate your input.

Answer 1

"We have a mandate from the management to lower our bug count, regardless of severities."

Some of this is not possible. At the same time, it is possible to pull together a plan that will reduce the bug count found by QA.

Software is going to have bugs, period. That is the nature of the beast. It is typical to spend 40% or more of any project in QA and fixing bugs before and after release. Considering the totality of any project, some projects spend up to 90% of the budget on finding and fixing bugs. Recently, a bug was found in a commonly used library that had been there for 8 years.

If you have to reduce the bug count found in QA, then most of what QA does has to be moved into the development team. That means that the developer has to work with QA to define the testing that has to happen before submitting to QA. The developer has to do the QA first.

One type of development that reduces the bug count in QA is called "Test Driven Development", or "TDD". It adds a lot of development costs as the developers have to build test beds first and then make sure that the code passes those tests before going to QA. But it does reduce the number of bugs caught by QA.

Another technique could be to have a group review every bug report from QA asking how this bug could have been caught by the team before submitting to QA. This could be a way to help educate the team on techniques to catch bugs, identify patterns of failures and ask for ways to prevent those patterns, and build a plan that could be presented to management on how to change the process so as to reduce the bug count.

So, you can present to management a plan by which the developers go through training on how to do a new process, a change in how stories pass through the development team with perhaps a second developer building the test plan and test bed, and supervisory changes where the supervisor is monitoring how the tests are being built and run during development. There might be a need for follow up training as many people need repeat training to get proficient in a new way of working. This plan will greatly impact the speed at which stories are completed but will reduce the bug count.

In essence, bug count can be reduced, but at a cost. Does the management really want to pay the cost?

Answer 2

We have a mandate from the management to lower our bug count, regardless of severities.

First you must decide if management actually wants higher quality code, or just wants a lower bug count.

If the former, you must study why you have so many bugs now and then attack the real problem. This isn't simple or quick, and may not be cheap, but it can provide real long term benefits. Without first understanding the problem, you are unlikely to fix it.

If the latter, then there are many strategies for gaming the bug count.

• argue about what actually counts as a bug
• release much less code in each story
• spend less time developing and more time having developers test before release
• try to get QA to "pre-test" everything (without writing any bug reports) before formally releasing it into test
• don't release anything until QA has no testing time remaining (this was a very effective tactic used by one developer where I worked)
• scale the QA team way down so they do less effective testing (I once worked at a company where one dev team tried to get rid of their sole QAer)
• etc, etc.

Answer 3

Some long term and authentic strategies may be (IMHO):

1. Work with the project managers to ensure that they give developers/testers enough time to develop high quality products for each milestone, and still meet the overall project deadline for the customers. Often, the need to deliver products really fast (without having enough time to write good code and test carefully) leads to more bugs in the software.
2. Hire talented and dedicated developers / testers / project managers.
3. Hire enough developers to ensure that they are not overloaded with too much work. This way, developers would have enough time to write high quality code and time to carefully test the code. The similar strategy applies to testers and project managers.
4. Pay the high salary or at least good salary to keep talented employees at the company.
5. Make sure that the same bugs are not reported more than once.

Answer 4

“Lower bug count” is an irrational target. “Time until release of a shippable product” is a rational target.

“Lower bug count” is easily achieved by having an informal agreement with QA that problems are reported in person, not as official bugs, in exchange for whatever favours.

Lower bug count is also achieved by excessive developer testing. There is an optimal amount of developer testing, basically at the point where QA testing with bug reports is cheaper than more testing in development. When you reach that point, you can increase total cost while reducing bug count. (Of course too little developer testing also costs).

Answer 5

Does your management also mandate how many lines of code you have to write every day? Cause if they do, maybe you should look for new management.

Bug count is not a metric for anything. Bugs are not something you can avoid with carefully crafted strategies. They are artefacts, results of imperfect work, discovered while analysing said work in a dynamic environment. Reducing the number of bugs regardless of severity is a moving goalpost, it's not a SMART goal for any development team. If anything, you want the riskiest bugs to be discovered as early as possible. The more bugs your testers discover, the less risky it is to launch your product.

This management directive will only lead to two things: a lot of setbacks in implementing stories or fixing issues, and the apprehension of the testing members to submit new issues. Both are worse than a spiralling defect count.

Small personal anecdote here: a long time ago one of my test managers decided it was a good idea to set a minimum number of bugs testers had to add before packing up and going home. After working overtime for 2 weeks to get a project out of Beta. Our entire team (6 testers) filed that specific number of issues (we just got creative with our reproduction steps and interpretation of requirements) then we all quit. Don't use bug counts as metrics.

TL;DR: Your management is wrong to ask this of you, helping them achieve that objective would be counter productive to the project goal, please push back against such asinine requests.

Answer 6

We have a mandate from the management to lower our bug count, regardless of severities.

There are lots of different ways you can do this, most of which are pretty simple. Things like:

• Stop reporting new bugs.
• Set a quota for how many bugs each developer can report, and require them to add some new feature requests once they reach the end of the quota.
• Add lots of new (low priority) feature requests to reduce the ratio of bugs : feature requests.
• Create much broader and higher level issues that cover multiple bugs at once.
• Create a new issue called "Outstanding Bugs", and add any new bugs you find into there.
• Set up a separate issue tracker and use that to track bugs.

And if those suggestions sound stupid to you, then you need to push back against your management and explain to them why "number of reported bugs" is a terrible metric.

Because ultimately this isn't a software development problem: it's a problem with someone non-technical setting a stupid target without thinking about what they're actually trying to achieve.