51

I read an idea for increasing productivity in a company. It went like this:

Have a certain fund that will be a bonus. Say $100,000. For each tangible bug found, the testers get paid $5 - $15. Whatever's left over at the end of the month/year goes to the developers.

It seems like a wonderful enough idea in theory, though I'm not sure how well it will work in practice. The obvious consequence of this is that it promotes antagonism between developers and testers. The bug business becoming a zero-sum game.

Is the antagonism a bad thing? How will it affect productivity, and more importantly the organisation? Personal experience will be highly appreciated.

P.S: I'm not in any managerial position (still in college) though I do have plans for a few software startups when I graduate.

  • 68
    The core idea here is that it's better to place a team in competition than collaboration. And no, even when you ignore the practical problems with this plan, that's a recipe for failure. – Nathan Cooper Dec 11 '16 at 11:29
  • 8
    maybe for the first run, then I'm going to sit there knowing my colleague got a massive bonus and I didn't even though we worked equally hard, that's not going to do my morale any good – Toni Leigh Dec 11 '16 at 13:26
  • 18
    I up-voted the question because it is a really important concept to get right. But, please, take the advice of all of these responders who have a lot of diverse experience and backgrounds, and who all agree that this is an idea with MANY harmful side effects that could have a far more negative impact than any good that might come from it. – Kent A. Dec 11 '16 at 13:55
  • 21
    Daily WTF has an excellent story of something similar to this and you can see what happened: The Defect Black Market – Anketam Dec 11 '16 at 16:57
  • 10
    Mandatory Dilbert reference: dilbert.com/strip/1995-11-13 – Val Dec 11 '16 at 20:40

11 Answers 11

110

I read an idea for increasing productivity in a company. It went like this:

Have a certain fund, that will be a bonus. Say $100,000. For each tangible bug found, the testers get paid $5 - $15. Whatever's left over at the end of the month/year goes to the Devs.

It seems like a wonderful enough idea in theory, though I'm not sure how well it will work in practice. The obvious consequence of this, is that it promotes antagonism between Devs and testers. The bug business becoming a zero sum game.

Is the antagonism a bad thing? How will it affect productivity, and more importantly the organisation? Personal experience will be highly appreciated.

(I cringe when I read "motivational" schemes like this.)

Perhaps the author of this idea defines "productivity" differently than I would.

The goal of most companies producing software is to make money, and perhaps maximize the amount of money they make. The goal of a bug bounty system is less clear, and certainly has nothing to do with productivity. A lot of time would be spent trying to get bonus money, and little time would be spent trying to ship the software (which is how the company makes money).

Imagine a company having one developer and one tester.

If you were the developer, your optimal strategy would be to withhold all software from the tester until there were no bugs remaining at all, or until there was no time in the schedule for the tester to find any.

I worked at a company that rewarded developers with praise, rather than money, and that was exactly the strategy used by one developer. Builds were released to QA every 2 weeks. And every other Friday, we had a full-team meeting where the bug count in the current build was announced. If a developer ever had zero bugs, that developer was singled out and praised for doing a great job.

One developer decided to game the system. She would hold the build (always having reasonable excuses) until just before the weekly meeting. She then released the build, the bug count report was run, and "magically" she had a zero bug count just in time for the meeting.

While she was a good developer, her code wasn't any better than most others. Her cleverness was in manipulating the system to her advantage.

I currently work in a shop that punishes Developers for the number of bugs attributed to their work during a project. They are expected to "improve" their bug count from the prior year. This is part of their MBOs, and part of what goes into their annual bonus payout.

Unsurprisingly, they have taken a long time to produce the first testable build for QA. And they asked that QA spend a lot of time testing in the Dev environment (where bug reports cannot be written). They have been given every chance to produce fewer bugs in the reporting system, and hence to maximize their bonus.

The Product Manager has even decided to change many bug reports to "enhancement requests" so as not to impact developer MBOs. His argument was "well, the developers haven't fully developed that feature, so they will enhance it when they have time".

If I were getting paid "by the bug" I could find a ton of bugs quite easily, no matter how good the developer. (I've been doing this work for almost 30 years). I'd focus initially on the low hanging fruit and skip anything that was time consuming at all. My bug reports would be minimal and not very helpful - basically whatever it took to enter the bug report in the system and get the cash.

The result would be a lot of superficial bug reports that were "just tangible enough" and the software would inevitably be left with some major critical bugs. I can't imagine the system would ever ship.

The developers would focus all their time and attention on new code. There would be absolutely no financial benefit in fixing any bugs that were already found. And they would be given incentive to produce tiny, insignificant releases. If they produce a release containing only a single line of perfect code, they get a $100k bonus! Why add any tricky features at all?

Both teams would argue vehemently about each and every bug report and if it was truly a "tangible" bug or not. (That's a nice and fuzzy term. I'd love to hear a team try to define it concretely). These sorts of arguments don't set the stage for anything I'd call productivity.

And neither testers nor developers would spend any time on anything else. No meetings, no documentation, no customer support, no helping others, no prep for shipping. Hey, if it were important, there would be cash attached to it, right?

And this last part is significant. For knowledge workers, often attaching cash bonuses to tasks they intrinsically feel are important is a big disincentive. If you want a greater understanding of the kinds of dysfunction that can arise from these sorts of incentive systems, I highly recommend you read Measuring and Managing Performance in Organizations by Robert D. Austin.

In short, this is a terrible idea.

Most good software companies recognize the folly of such a plan and try to stay away from this sort of system. Most software companies understand that releasing software with zero bugs isn't a realistic goal, and that it's more important to release software in a timely manner.

  • 43
    'Enhancement requests' is a good point here — as a developer, if you deduct money from my pay for every bug, it makes it in my interest to fight tooth & nail over the spec: "the story never said the Date of Birth field couldn't be in the future. This isn't a bug, it's an additional story." In addition to rewarding devs for working slower and for being less helpful to QA team, you're also incentivising them to ignore common sense and needing everything documented by the Product Owner/Analyst to the Nth degree. – anotherdave Dec 11 '16 at 14:26
  • 2
  • 1
    Adding to all of this, I can imagine there would be an hanging pole outside the developers' office for anyone who dares refactoring old code, leaving the codebase in a complete mess because "If it isn't buggy, don't even try to touch it!" – BgrWorker Dec 12 '16 at 8:11
  • 2
    @aroth - I'm gonna write me a new minivan this afternoon! dilbert.com/strip/1995-11-13 (be careful what you pay for - you won't get what you might expect) – WorkerDrone Dec 12 '16 at 13:51
  • 1
    I think @aroth was spot on with his remarks. Albeit the OP was not likely to do well with his setup, this could be framed better. Even this posts comments could easily be taken out with a simple framework (such as only major/critical bugs receive bounty, pre-existing bugs only, and not making it a competition between devs and testers). – Sh4d0wsPlyr Dec 12 '16 at 15:29
129

It seems like a dreadful idea. Here's a few things that will happen, in addition to your developers and testers starting to hate each other and yourself for introducing this:

  • Everyone will focus on low hanging fruit. This means that QA will start reporting all sorts of stuff that's actually fine but might be construed to be "buggy" in hopes of getting paid, while Devs will focus all their work on making sure there are no obvious bugs and a lot less on making sure there are no complex, structural bugs.
  • Some people will gang up and a dev will intentionally introduce a ton of bugs, then send it to a specific QA to "find" them, then split the cash.
  • Some of your employees will be insulted because you think they only value their job if they get a bonus for it. They will probably work less hard and produce more junk because of this.
  • Communication between members of the team will break down because of increased antagonism. It's now actually a plus for the devs to not help QA do their job better, because any bugs that go into production unnoticed means they get paid more.
  • Devs will dislike each other, because every bug one dev introduces will cost all of them money.

This is just off the top of my head. Don't ever try to motivate programmers and QAs with money, it only turns out terrible. Their jobs run on intrinsic motivation.

Also, please have a look at this animated TED-talk about drive and motivation, as it explains much better why any setup that involves motivating smart people with money will fail terribly:

https://www.youtube.com/watch?v=u6XAPnuFjJc

  • 19
    +1 I think it's also worth stating that you're in a market for developers and are competing with other firms. The negative externalities from this proposition will lead to a situation where the good developers leave and the lemons are left. QA will be pleased as the change in the composition in the quality of the software engineers will lead to larger bonuses for them under this scheme. This is one of the most destructive ideas I've come across. – Wintermute Dec 11 '16 at 10:51
  • 3
    +1 This is likely to breed hostility among the team that could even go beyond the tech team and QA. Developers will argue about who introduced a bug and whether it's a bug in the first place or possibly a result of badly written specification, bad design etc. – Llewellyn Dec 11 '16 at 10:57
  • 4
    The colluding argument is very real. I read about a case where devs served testers some cool loopholes and caveats, they split the bonuses. I think the scheme was discovered because they went too big, too often, i.e. super hidden bugs started appearing and discovered at a suspicious rates. – luk32 Dec 11 '16 at 22:01
  • 4
    "QA's will start reporting all sorts of stuff that's actually fine but might be construed to be 'buggy'" The QA staff where I work do this anyway. Half the "bugs" the QA's open on my project are design advice ("Put a loading spinner on this page"), grammar corrections (which are usually wrong), and things that are working as intended but not according to the QA's expectations. A few times we've nearly overlooked possibly world-ending bugs because of this. If they were getting paid bonuses for this, they'd flood us with spurious garbage bugs. – Torisuda Dec 12 '16 at 1:28
  • 3
    @JoeStrazzere There's a step missing. The QA team will also be stripped of the people that value quality over reward. The remaining QA members will be happy with situation, since they get more low hanging fruit which means more money for them. – Jasper Dec 12 '16 at 13:29
21

This makes as much sense as having a ship's crew split into teams, those who do propulsion and those who navigate, compete in a race, on the same ship.

I refuse to have an antagonistic relationship with my testers. I value them. They make me a better coder.

I also respect the creativity their job demands of them. Which is why I think money is the wrong motivator here. Studies have shown that unless a job is practically mindless, cash incentives actually, measurably, slow people down.

Creative work isn't best motivated by money. Its best motivators are:

autonomy – the desire to direct our own lives
mastery – the urge to get better, or develop skills
and purpose – the need to do what we do for reasons bigger than ourselves

That's right, choices, opportunities to improve, and teamwork would work better than money.

QA is a creative job. The task really is to think of what the developers didn't think of. This is why QA should automate. Once a test is thought of it shouldn't be "performed" again and again like a Broadway play. It should be automated so QA can stop thinking about it and think about the next test. QA should be filled with your most talented developers. Because they're trying to out think your other team of developers.

Some don't think so. Some think of QA as a dumping ground for less talented developers. If you've been doing that, your priorities are backwards. Challenge your best developers to modernize your testing and make sure people know that QA is where you put the best.

If that money is still burning a hole in your pocket use it on training or if need be, severance packages.

We don't do mindless work here.

14

Have a certain fund, that will be a bonus. Say $100,000. For each tangible bug found, the testers get paid $5 - $15. Whatever's left over at the end of the month/year goes to the Devs.

That's horrible. For all the reasons in the other answers and for the fact that it does not pass this very simple test:

  • What if all your employees are great and no bugs are found?

    Production works perfectly and all the money will go to the DEVS

  • What if all your employees are crap?

    Production just burned to the ground because of the bugs, but hey, who cares, the money will go to the DEVS.

Even if money were a motivator in our business, this would be horribly wrong.

Take the money and hire somebody who is really good at writing specs and planning projects with manageable deadlines. Both DEV and QA will be way happier than any money could ever make them.

  • Good point to suggest alternative uses for $100,000 fund. There are probably a few other choice uses of this much money to make a team of devs and QAs more productive. Although I think it would go off-topic to start brainstorming them. – Neil Slater Dec 11 '16 at 12:11
  • "Production just burned to the ground because of the bugs, but hey, who cares, the money will go to the DEVS." - in theory that's the incentive for QA to do better, because they will get paid for not letting production get burned to the ground. – user253751 Dec 11 '16 at 23:11
9

It may be apocryphal, but I was given a similar example in the 1980's when covering the "Law of Unintended Consequences" as part of a BPR course.

The example concerned a factory production line where the quality control department were incentivised by how many rejects they made. The production department was similarly incentivised according to how few rejects they produced.

The net effect was that quality control rejected more products than previously and production took longer to produce "perfect" items, so overall output went down while costs went up due to the incentive payments. Quality was unaffected.

5

It works badly. I haven't seen it with devs and testers. But the justice department in NZ at one point rewarded periodic detention wardens for each detainee they breached. It went from one breach in a bad week, to some detainees getting 6 breaches in a day and ending up in prison over it. Eventually a warden got hurt.

I doubt it would go that far since it's not the same amount and less volatile people (I would think), but it breeds bad blood between the groups who are already at odds due to their roles.

  • 9
    I have no clue what it means to breach a detainee. – emory Dec 11 '16 at 18:20
  • 5
    I'm guessing 'breach a detainee' means 'make an official record that a detainee breached a rule' – bdsl Dec 11 '16 at 22:59
  • @emory My understanding is that it was an official writeup and if a detainee got 3 within their sentence, they were arrested, taken back to court and resentenced to a prison term. – Kilisi Dec 12 '16 at 9:18
3

Other answers have already sufficiently stated that your idea is bad TM. So I won't further repeat that, and instead mention what you'd need to change to improve the idea.

You're trying to attach money to what's ultimately number on a paper. That number won't create any money. To make things worse, the number you want to use is mostly random: A product that never had any bugs suddenly has 50, all of them typos in the documentation. A bug that messes up the highlight color of 200 items becomes 200 bugs. As soon as you ask some people to make the number magically go up, that number is a completely useless imaginary number. And on top of it all you'll waste tens of thousands of dollars in wages with absolutely pointless meetings where people argue what is and what isn't a bug.

If you want to attach bonuses or other rewards to numbers on a paper, these numbers must be directly linked to actual money earned by the company - one common example is bonus being linked to profit in a trading company.

3

As has been suggested in the 'cupcake' comments under your question, some forms of gamification can work. How about assigning a trophy for the 'Best Bug' found during a certain period, assigned by votes from both teams together?

In a healthy development environment there is a shared 'awe' between testers and developers about people (usually the testers) finding those pesky bugs that can only be reproduced under certain conditions, about that intermittent bug that has been pestering you for months, about something very simple that a developer overlooked, etc.*

The trophy should be something very simple, because it's not about its intrinsic value: a small sign, a cupcake, a bar of chocolate.

* That last example is not about finding a bug, but causing a bug. In that case the prize means friendly mockery. You do need a culture for that where your colleagues recognize that everybody makes mistakes.

2

I think the outcome would be detrimental. It's both against common sense practices of both developing and testing.
Zero bugs found doesn't mean that software is good. It could be to complex to test or QA could skip some testing. Even zero bugs found, extensive testing process and great quality of code doesn't mean that product is good. If the requirements were messed, the software can be terrible for client.

Playing against that system could be extremely easy.
For devs: late builds. For testers: concentration on trivial bugs.
With late builds, early testing is impossible. Little time for testing and gratification for number of bugs? Testers concentrate on trivial bugs, like misspelled labels or some non-critical cases.
If you want to know if the product is good, just ask your client.

2

Long story short. Developers and testers are working on same boat - your company.

It is easy to critisize developer for a bug they produce but it is hard to reward them for bugs they haven't produced.

If the bug made it to final build and customer reported it who was to be blamed? Developer for writing the bug or tester for not catching it?

This causes the tension between developers and testers by default. Your idea puts aditional tension between the two. You really do not want that.

If you have friendly workspace and want to reward the bug-catching split the budget to everyone fairly, make a list of developers and for every bug the developer shall pay $1 to the bank. Organise the Xmas party and use the bank to reward whole team. Ensure, it is more fun than reward/punishment and that nobody takes it (too) serious. Reward both the "worst" and "best" among developers and testers.

1

Honestly, the incentive at best seems useless and at worse harmful. If you have a devoted QA team you already have set aside funds to find bugs as that will be a big part of their job description. There won't be much added antagonism between the two as there will be more toward the policy itself. If you are actually worried about the problem that much to throw money at your current employees with strings attached, it would be better to hire someone where you need them in either QA or dev.

Where this could go really wrong is on the dev side is that you'll see a drop in releases for fear they'll get paid less because of new bugs and on the QA side you could see much longer review phases due to testers wanting some extra pay. Not saying this will happen, just that it's a possibility.

Nobody wants bugs in their final product, but the will happen. As a manager there are much better ways to make sure less bugs exist in the code such as having realistic goals and time tables and allowing facilitation for better development like code reviews.

  • But they aren't getting paid less. I'm not deducting their agreed upon salary. I'm giving them a bonus based on how many bugs were absent in their code. If longer review periods mean a little better quality, why not. There should be a policy imposed timeframe for review though. – Tobi Alafin Dec 11 '16 at 8:54
  • 9
    @TobiAlafin: They're getting paid less than they would if no bugs were found. It's human nature to think of this as a loss. – Llewellyn Dec 11 '16 at 10:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.