7

I am using a mechanism that tracks throughput (so that we can estimate tasks acurately for each individual on projects). Having this data could cause issues (i.e. with people motivated to make the numbers as best they can by rushing, or just plain lying about time spent).

Has anyone run into this? How would you deal with this information being readily available to the team and management? How would you prevent someone from using this data for improper purposes and or abusing the process in order to get their numbers up?

  • 4
    You also need to track quality of work if you don't want through put to impact on it. This may be in forms of - defect rates, rework rates etc... – Oded Sep 19 '12 at 15:39
4

People will only game the system if there's a benefit to doing so.

If you're tracking throughput to make future estimates vaguely accurate then there is a cost to gaming the system. Make it look like you're doing more and you will be expected to keep up that record. Make it look like you're doing less and the whole team looks bad.

BUT if you start rewarding people who inflate their personal statistics, there is suddenly a benefit.

So, if you don't want people gaming the system, don't reward them for doing so. Simple.

If, however, you WANT to compare people statistically and reward your best performers then you need to look at everything: Quantity, quality, distractions to other people, helping other people, encouraging other people, time worked (if someone is overworking themselves before a review, are they going to be able to sustain that). Anything that has an effect on the team's productivity, measure it. How easy this is depends very much on your industry.

  • Exactly - for the record, tracking of productivity AND quality is pretty common in some cases - most defense contractors do it as part of CMMI practices. And it's just what @pdr says - it's considered part of your job to be productive, and maintain quality, and be an asset to the team. Awards, incentives, and raises are holistic. The numbers are usually used in aggregation to answer questions about future projects, things groups could get value from improving, and lessons learned. – bethlakshmi Sep 19 '12 at 18:25
  • -1: So, if you don't want people gaming the system, don't reward them for doing so. Simple...Anything that has an effect on the team's productivity, measure it... Wow. Either you just totally contradicted yourself or you're insisting that the scorekeepers have über-responsibility to keep people from gaming the system (and I'd venture to say that this would be either cost-prohibitive, futile, or impossible). – Jim G. Sep 27 '12 at 0:26
  • @dbobrowski: I can't believe that this is the accepted answer. You and pdr need to read and learn from the other answers. – Jim G. Sep 27 '12 at 0:30
  • @JimG. Well, I did upvote one of the other answers. However, I think you've misunderstood mine. I think there are some industries where you can reasonably measure one person against another. But I agree that, in others, it is impossible or not worth it. That was the point of my last sentence. – pdr Sep 27 '12 at 1:38
5

The unfortunate thing about "metrics", "KPI's" or whatever you want to call them is that people are all too willing to uncritically accept such measurements as "the truth" just because they're numbers. Moreover, far too much emphasis is placed on the collection and reporting of these dubious figures and relatively little effort is put into their intelligent interpretation and how to effectively use these measurements in practice.

A simple number is much easier to present and digest than nuanced sentences, comparative case studies, and historical background. This plays into the desire of management to make crisp, quick decisions based on "objective data". It understandably feels better to make black and white decisions or judgements and not have to be mired in a fog of "grey area" indeterminancy. Unfortunately, reality itself often is a "grey area" and it is not always possible to make truly informed decisions based on data, let alone data that isn't even valid.

As an example, in manufacturing, it is not uncommon for organizations to practice "cargo-cult" versions of six-sigma or lean methodologies. In these places, a superficial effort is made to collect easily measured data and then cherry-pick the data differently to report the desired outcome. This leads to people "gaming the system" for their own benefit while the true outcomes fail to change in a controllable way.

The way to avoid this problem is to cultivate a culture where decision-makers are willing to spend significant effort to look at data intelligently and critically. Such people would not blindly accept what their "business intelligence dashboard" tells them and would smirk with skepticism upon hearing the phrase "high visibility into productivity."

2

I ran into something like this in a previous job. A point system was created for finding issues, completing tests, and other QA tasks. Bonuses were awarded with the top point earner each month earning $500, the second earning $250, and the third earning $100. Within three months, the system was discarded and deemed a failure. We pumped out more work, but the consequence was a drastic dip in quality due to speeding through tasks to get more points as well as a plethora of illegitimate bugs logged so testers could garner points.

Even if the employees do not have access to the metrics you are collecting, once they are aware that they are being recorded, the likelihood of behavior changing for the worst is high. No employee wants to look bad in a direct numerical comparison with their co-workers, I suspect if dishonest behavior is not deliberate, there is a subconscious pull to appear to be doing a lot of things.

I'm not sure if you can work a logging mechanism into whatever software you use to track workflow, but doing that without letting the employees know may be your best bet to get honest results, but that comes at the cost of what some may feel is "spying behavior" on the company's part (even if it really isn't, since a company has every right to gain metrics on their employees work)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.