1

Looking for ideas to ensure bugs count kept low say about 40-50% of total stories submitted to testing, here are the steps we tried

  1. Before QA, I ask the story owner to demo the work
  2. Code review, hoping to catch bugs before going to QA, even thought code reviews are for checking code compliance with our standards, unit tests etc..
  3. Ensuring that devs understand the story and the acceptance criteria

We have a mandate from the management to lower our bug count, regardless of severities. Appreciate your input.

6
  • 2
    Why are your code reviews only doing what a linter can do? Aug 27, 2022 at 5:49
  • 3
    And you need to detail the exact nature of the bugs? Is the code doing what the author intended, just they misunderstood the requirements? Or is the code misbehaving? Aug 27, 2022 at 5:51
  • 16
    This feels very much like a question for the software engineering site rather than workplace.
    – Saes
    Aug 27, 2022 at 6:41
  • 1
    What is your role in this process ? developer ? team lead ? manager ? or PM ? Aug 27, 2022 at 22:21
  • 7
    Why are these bugs in the first place? Bad specifications? Inexperienced programmers? Moving goalposts? Identifying and then fixing the causes will most likely be the most efficient approach. Aug 28, 2022 at 14:09

6 Answers 6

6

"We have a mandate from the management to lower our bug count, regardless of severities."

Some of this is not possible. At the same time, it is possible to pull together a plan that will reduce the bug count found by QA.

Software is going to have bugs, period. That is the nature of the beast. It is typical to spend 40% or more of any project in QA and fixing bugs before and after release. Considering the totality of any project, some projects spend up to 90% of the budget on finding and fixing bugs. Recently, a bug was found in a commonly used library that had been there for 8 years.

If you have to reduce the bug count found in QA, then most of what QA does has to be moved into the development team. That means that the developer has to work with QA to define the testing that has to happen before submitting to QA. The developer has to do the QA first.

One type of development that reduces the bug count in QA is called "Test Driven Development", or "TDD". It adds a lot of development costs as the developers have to build test beds first and then make sure that the code passes those tests before going to QA. But it does reduce the number of bugs caught by QA.

Another technique could be to have a group review every bug report from QA asking how this bug could have been caught by the team before submitting to QA. This could be a way to help educate the team on techniques to catch bugs, identify patterns of failures and ask for ways to prevent those patterns, and build a plan that could be presented to management on how to change the process so as to reduce the bug count.

So, you can present to management a plan by which the developers go through training on how to do a new process, a change in how stories pass through the development team with perhaps a second developer building the test plan and test bed, and supervisory changes where the supervisor is monitoring how the tests are being built and run during development. There might be a need for follow up training as many people need repeat training to get proficient in a new way of working. This plan will greatly impact the speed at which stories are completed but will reduce the bug count.

In essence, bug count can be reduced, but at a cost. Does the management really want to pay the cost?

1
  • You can add Unit test, integration test, CI, architecture review, pair programming, mob programming, DDD, BDD, and so on.
    – JayZ
    Aug 30, 2022 at 13:22
4

We have a mandate from the management to lower our bug count, regardless of severities.

First you must decide if management actually wants higher quality code, or just wants a lower bug count.

If the former, you must study why you have so many bugs now and then attack the real problem. This isn't simple or quick, and may not be cheap, but it can provide real long term benefits. Without first understanding the problem, you are unlikely to fix it.

If the latter, then there are many strategies for gaming the bug count.

  • argue about what actually counts as a bug
  • release much less code in each story
  • spend less time developing and more time having developers test before release
  • try to get QA to "pre-test" everything (without writing any bug reports) before formally releasing it into test
  • don't release anything until QA has no testing time remaining (this was a very effective tactic used by one developer where I worked)
  • scale the QA team way down so they do less effective testing (I once worked at a company where one dev team tried to get rid of their sole QAer)
  • etc, etc.
1
  • 6
    If you work a lot, you make a lot of mistakes. If you work less, you make fewer mistakes. If you don’t work at all, you make no mistakes. If you make no mistakes, you get promoted.
    – gnasher729
    Aug 29, 2022 at 8:35
2

Some long term and authentic strategies may be (IMHO):

  1. Work with the project managers to ensure that they give developers/testers enough time to develop high quality products for each milestone, and still meet the overall project deadline for the customers. Often, the need to deliver products really fast (without having enough time to write good code and test carefully) leads to more bugs in the software.
  2. Hire talented and dedicated developers / testers / project managers.
  3. Hire enough developers to ensure that they are not overloaded with too much work. This way, developers would have enough time to write high quality code and time to carefully test the code. The similar strategy applies to testers and project managers.
  4. Pay the high salary or at least good salary to keep talented employees at the company.
  5. Make sure that the same bugs are not reported more than once.
2

“Lower bug count” is an irrational target. “Time until release of a shippable product” is a rational target.

“Lower bug count” is easily achieved by having an informal agreement with QA that problems are reported in person, not as official bugs, in exchange for whatever favours.

Lower bug count is also achieved by excessive developer testing. There is an optimal amount of developer testing, basically at the point where QA testing with bug reports is cheaper than more testing in development. When you reach that point, you can increase total cost while reducing bug count. (Of course too little developer testing also costs).

7
  • 1
    I once got 40 bug reports where X didn’t work, and as a result A failed, B failed, C failed and so on and so on.
    – gnasher729
    Aug 29, 2022 at 8:33
  • @JoeStrazzere punishing the team for management arrogance, the classic.
    – BoboDarph
    Aug 30, 2022 at 6:39
  • 1
    @JoeStrazzere Metrics are not the problem it is management not understanding what they are looking at.
    – Joe W
    Aug 31, 2022 at 0:53
  • @JoeStrazzere There is nothing wrong with relying on metrics as long as you understand what the metrics really mean and which ones are useful versus which ones are not.
    – Joe W
    Aug 31, 2022 at 12:11
  • @JoeStrazzere It is obvious we disagree but there is no way getting around needing metrics to properly evaluate performance and it is a matter of making sure you are using the metrics properly.
    – Joe W
    Aug 31, 2022 at 15:42
1

Does your management also mandate how many lines of code you have to write every day? Cause if they do, maybe you should look for new management.

Bug count is not a metric for anything. Bugs are not something you can avoid with carefully crafted strategies. They are artefacts, results of imperfect work, discovered while analysing said work in a dynamic environment. Reducing the number of bugs regardless of severity is a moving goalpost, it's not a SMART goal for any development team. If anything, you want the riskiest bugs to be discovered as early as possible. The more bugs your testers discover, the less risky it is to launch your product.

This management directive will only lead to two things: a lot of setbacks in implementing stories or fixing issues, and the apprehension of the testing members to submit new issues. Both are worse than a spiralling defect count.

Small personal anecdote here: a long time ago one of my test managers decided it was a good idea to set a minimum number of bugs testers had to add before packing up and going home. After working overtime for 2 weeks to get a project out of Beta. Our entire team (6 testers) filed that specific number of issues (we just got creative with our reproduction steps and interpretation of requirements) then we all quit. Don't use bug counts as metrics.

TL;DR: Your management is wrong to ask this of you, helping them achieve that objective would be counter productive to the project goal, please push back against such asinine requests.

0

We have a mandate from the management to lower our bug count, regardless of severities.

There are lots of different ways you can do this, most of which are pretty simple. Things like:

  • Stop reporting new bugs.
  • Set a quota for how many bugs each developer can report, and require them to add some new feature requests once they reach the end of the quota.
  • Add lots of new (low priority) feature requests to reduce the ratio of bugs : feature requests.
  • Create much broader and higher level issues that cover multiple bugs at once.
  • Create a new issue called "Outstanding Bugs", and add any new bugs you find into there.
  • Set up a separate issue tracker and use that to track bugs.

And if those suggestions sound stupid to you, then you need to push back against your management and explain to them why "number of reported bugs" is a terrible metric.

Because ultimately this isn't a software development problem: it's a problem with someone non-technical setting a stupid target without thinking about what they're actually trying to achieve.

4
  • I'd ask managment what they are trying to achieve, rather going straight in with the attitude it's a terrible metric and a stupid target. In my experience, there's normally a reason why they are trying to do something like this ("the client's are leaving us because there are so many bugs", "we can't schedule any new work because we're fixing so many bugs") Aug 28, 2022 at 8:01
  • 2
    The most stupid aspect of what OP is being asked to do is the "regardless of severity" and it's this that needs pushing back on hardest. Without a clear definition of good enough then QA will be forever chasing their tail trying to reach perfection. Meanwhile, management are valuing superficial stuff at the same level as critical faults, and that leads to everyone gaming the metric either focusing on finding and fixing what's easy, or hiding bugs hoping nobody notices them. Aug 28, 2022 at 10:22
  • 1
    You might replace “severity” with “annoyance”. If your primary objective is “reduce customer complaints” then fix what customers complain about and make them happy.
    – gnasher729
    Aug 28, 2022 at 14:59
  • And I’m happily using software that crashes sometimes, maybe once a day, and each crash costs me ten seconds of my time, with no work lost. So what is critical?
    – gnasher729
    Aug 28, 2022 at 15:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .