How do I ensure fair performance ratings in the stack ranking system?

Question

I am a lead programmer and moved jobs recently. In my new assignment, I am a technical manager for a large team. I have 5 people under me, each of whom have 6 people under them. So essentially I oversee 30 people and will be responsible for their annual performance review. We might hire more folks so I might end up having 40 folks under me. I am a very people-centered manager with great mentoring skills, zero-tolerance for politics and excited about the opportunity. I definitely plan to spend invest time guiding every single member of the team and take them to the next level.

The unfortunate thing is we have the Stack Ranking system. This means out of the 30 people, maybe 5-6 people will rank top of the stack while the rest will receive very average reviews. To ensure that the very best get top rating, I am thinking the only way to eliminate subjective and non-tangible factors is to have a well-defined scoring system that gives us reasonable basis as to why John Smith is ranked ahead of Jack Doe.

For example, each employee's score will be cumulative based on:

1. Customer rating
2. Performance on defined tasks
3. Participation beyond assigned tasks
4. Cross-training rest of the team
5. Volunteering/Evangelism/Hackathons/Public Speaking
6. Getting things done
7. New Ideas

This would ensure a range of scores - ranging from 30/100 to 90/100 maybe.

Another idea I can think of is instead of a relatively rating A's reportees A1, A2, A3 and A4 (where A1 is an experienced member and A4 is a relatively new member), we compare A4 with B4, C4 and D4(all are similarly experienced) while A1 is compared with B1, C1 and D1 (all similarly experienced). Then A4's position within A's team is determined, subject to how A1, A2, A3 fared when compared across their peers in teams B, C and D.

These are just my thoughts. I am very curious to know from experienced leaders how they tackled the negative side-effects of stack-ranking and managed to keep employee morale from sinking like a stone? Sure, I can vibe well with a number of employees but at the end of the day, I'm not the guy deciding their paycheck.

Answer 1

I worked at a place that changed to the Stack system after my second year. My boss hated it and fought against it, but in the end had to toe the line like everyone else. So I'm answering this based on what he did that engendered a lot of loyalty and respect from his reports.

The first thing he did is called a team meeting, to communicate what had happened and what it meant. This of course was met with a lot of groans and more than a little consternation. He let us know he had fought for a more fair system, but lost in the end. And as a good manager does, he left it at that - there was not bad mouthing the powers-that-be.

The second thing he did was explain what his plans were for balancing the unfairness of the new system. He wasn't going to be able to give his entire team stellar reviews; it would be impossible under the Stack system. There would be those that hit every one of their goals and float to the top of the stack naturally. There would be those who came close and would be compensated in other ways - small bonuses, time off, etc.

The third thing he did was follow through on every single promise he made. There were extended vacations that didn't get put on the books, small bonuses, team lunches, and office policy (casual clothing, remote work, etc.) that made up for the short comings of the ratings system.

The last thing he did was to continue trying to change the rating system all the way up to his last day in the office. The culture in the company became very toxic and his ability to buffer his team had all but disappeared. But he continuted to push at every opportunity for change and to make sure we were taken care of to the best of his ability.

The end result was half his team leaving with him. We didn't all go to work in the same office. We just knew that if he was leaving it was truly hopeless. And from what those who stayed behind had to say - we were right to leave. In an unfair system, he found a way to continue to take care of his team and engendered great loyalty and respect from all of us. We knew what he was up against and we knew he didn't have to be as creative as he was or to take as much heat as brought on himself, for our sake.

A mass exodus ended up being the catalyst for change at that company and they are still paying for that mistake, even to this day. Hopefully, you will have a happier ending and the execs making these decisions will not be as deaf as ours were.

My advice - do your best to be as fair as you can and give your reports the leeway they need to do the same for theirs.

Answer 2

Wow, I left this one, figuring by now someone may have a better idea that I did and I was really looking forward to reading that response.

Here goes mine.

Is your corporate culture slimy?

I doubt there are any lovers of stack ranking out there. No one likes being forced into a strict career defining paradigm that doesn't take into account any context of the people, the team, the work, or any other practical information.

With that said, I have seen some corporate cultures that were slimier than others about it. In the worst, stack ranking can become a witch hunt, no matter what the manager tries to do. And so it becomes more about finding the people to put on performance plans and less about thinking "is this team performing to it's expectations?"

In others, the forced stack ranking may well be a push to avoid a systemic nice-ness problem where for years the company has not held people accountable for self-improvement, and so the company has developed a culture of tolerating low performance that should be changed. A company like this may hold managers' feet to the flames for a few years but may back off when performance has measurably improved, and it may be willing to exempt teams or areas that are seen as already high-performing.

The point of asking the question is - at some level, if you are like most decent managers, you'll have trouble representing the values of a management culture that you view as slimy. It's one thing to be tough or demanding, it's another to feel that you are unfair. When you don't respect yourself for the decisions you had to make, find a better company.

Working in a tough (but not slimy) regime

Any system, no matter how brutal, will function better for people if they see the rules as clear and consistently applied. "Fair" is often subjective, but I see a lot more acceptance when it's consistent - no matter how ridiculous.

I usually get a lot of traction if I can explain something of the process and also of the criteria. Usually there will be a place where the doors are closed and what gets said in the room stays in the room. Being frank about the relative performance of the employees is the only way that managers can calibrate ratings - and no matter how you do it, that process isn't pretty. So there's always an art to being clear without being so detailed that you are getting into the sausage making.

Process

At some level, I get a certain amount of traction when I can describe the part of the process that I own. So that my folks know that at least from my level down, there's a clear set of tradeoffs and an effort is spent to seriously consider the ratings. I also get traction from admitting that there are parts of the process that I DON'T control and that my best hope there is to clearly and cogently explain my team's performance in a way that resonates in organizational terms.

One of the best points I ever heard about managers is - we are the translators from the technical/skilled world that does something to the organization at large which is communicating in a formalized alternate language that focuses on coordinating diverse efforts for the bigger goal. In that role, there is inevitably the need for the people who only live in one world to trust that I am correctly interpreting their value to the other world. The clearer I can be about how I do that - the more they will trust me.

The Criteria

Any performance ranking system will tell you that the criteria is best with it is specific, measurable, attainable, etc. (see SMART goals) and rating people against this nice clear criteria will certainly reduce arguments.

I would assert that it's not 100% possible. Particularly when you have to stack rank on an individual by individual level, then you will inevitably hit people who's very technical performance is equal, but there are some real standouts in the soft skills. I find soft skills very, very hard to measure, as the fact that all communication has a sender and a receiver means that two people are always a part of of the solution or the problem - so one person's mentor is another person's most-annoying-team-member. I wouldn't want to participate in a ranking system that ignored these soft skills, however, because it would mean that there is almost no incentive in helping someone if you don't get to have a stake in their success.

When it comes to soft skill criteria, I shy away from setting a specific number or value. I do weight demonstrated soft skills in terms of both impact and frequency, but it's an observed rating and admittedly flawed. When I give feedback in this areas, there's a alot "I want more of/less of..." and the "impact of XYZ behavior is ABC" and I avoid numbers.

Protecting your People

Assuming a not-slimy culture, I've seen cases where my management team has been able to adequately protect our people from painfully arbitrary stack ranked cut offs. There is much less morale-destruction if people can see and agree that people getting the poor ratings are really poor performers. And there IS a drop off where it's clear that a certain population isn't getting enough done. If I can get that line set accurately, most of the other points will work out OK. It's the fear that in a really arbitrary system, it's all just a game of musical chairs and anyone could be the next victim.

The argument usually centers around:

A - we perform better than other groups - so we should be able to rate our employees as better performers than the normal rules. This argument requires that you understand what the business thinks of as better and not what might be better in the technical sense. For example, if you answer customer questions as a primary function is the measurement # of really happy customers, or $ spent per question. Different businesses will see the metrics differently, but if you can show improvement in the beloved metric every year, then you can argue that you are exceeding the boundaries of the stack ranking and shouldn't have to lay of low performers.

B - prove that you manage people out prior to the stack ranking. If you fire/layoff people or get them to quit before you have to stack rank them, then you may be able to claim you don't need to rate so many as poor performers. For example, let's say you have 100 employees, 10 of them left during the year, and you have a stack ranking requirement that you put 10% of people on a performance plan. Then you can say that your 10 lost employees is your bottom 10% and the rest of the pool are the angels who kept the company together and they should all get a good or average rating. Let's face it - that one is a lie, in a place with poor retention, the top performers as likely (if not more likely) to leave as the poor performers, but since it lets you avoid the harshness of the system... go for it, but do put your truly poor performers on a plan.

Answer 3

First, I'm happy to see a manager concerned about this issue, but at the same time it's yet another tacit/implicit realization that there's something very wrong with the process.

TL;DR;

How do I ensure fair performance ratings in the stack ranking system?

You can't. You should avoid/abandon the process if you have the ability to do so.

After reading some of the other responses, I ultimately think they're just attempts to "patch" a fundamentally broken process and should be abandoned. Even if you don't currently have a "slimy" culture, as others mentioned (which is an important point), that's only marginally relevant.

Detailed Rationale

This is like asking "How can I make water stop being wet?". Wetness is a property of water just like unfairness is a property of the rank-and-yank system. Here're just a few reasons why I think this is the case.

The system:

1. is based on false assumptions;
2. expects predetermined results from the outset;
3. punishes real teamwork;
4. undermines trust;
5. is subjective and arbitrary;

I'll elaborate on each point below.

1. It's Based on False Assumptions

Another way of asking the question is to use rock-paper-scissors: "In rock-papers-scissors, which option gives me the highest/average/lowest win rate?"

The assumption that there is a better option is flawed. Since they all beat each other, no one option can be "better" or "worse" than the alternatives. They're good for different things and you'd expect managers to understand this, at least if they're worth their salt...

When someone comes along with the built-in assumption that there must be a distinction, and attempts to force one systematically, that's where things go sour.

The process implicitly assumes that:

1. employees, and only employees, are responsible for their performance;
2. real value is proportional to task visibility;
3. an employee's performance can be reduced to a number;
4. constant fear of getting laid off is a good "motivator" for improvement;
5. task context and complexity are the same for everyone;
6. "poor" performers actually exist in a team/organization;

These are a few examples of the fundamentally flawed assumptions at the foundation of this rank-and-yank process.

Point #1: This completely ignores any sort of impact management has regarding an employee's ability to perform.

I've seen managers be very poor at creating and/or prioritizing "important" tasks, only to see the engineers who complete the requested tasks get labeled as "poor" performers in ratings as a result (e.g. their works lacks "value").

In reality, the tasks were not as "important" nor had the "value" impact the manager(s) imagined them to have, yet employees working on them paid the consequences of poor leadership. The book, Leadership Gold, basically has a section that boils down to the following: If you want to evaluate a leader, look at the state of their followers.

Point #2: The level of visibility in your work does not determine the value produced by your work.

I've seen employees that did important/necessary, but less visible, work get under-represented and thrown under the bus during ratings for the sake of those that do more "visible" work or get better representation from their managers. Some employees will get "setup for failure" simply on the basis of the tasks they are given.

Point #3: An employee's performance cannot be reduced to a number in an spreadsheet; certainly not objectively.

You cannot have any level of "fairness" without complete objectivity and transparency, yet this system is anything but transparent. In my experience, even if someone tries to define "SMART" goals as others suggested, they're often defined at a high-level by management, rarely have a real impact in your actual day-to-day tasks, and may not get updated for several (6 - 12) months. (As an example, I'd always have "goals" about customer interactions, even though there was no way for me to interact with customers. It was nonsense.)

If you're aware of the day-to-day tasks your employees need to do, then use that as input for any SMART goals you create. If they're working on things you don't consider important, then you must take responsibility for the lack of direction openly and explicitly, apologize to your employee(s), be absolutely clear with them about it and that it's not their fault (they did not create/prioritize the work, you did), change their direction to tasks you consider important, and not penalize them in any sort of evaluation for this. Paraphrasing Sun Tzu, when there's confusion in the army, it is the fault of the general.

Point #4: Punishment is not a motivator for anyone on anything.

Although research is consistent with this, the fact that this is true ought to be self-evident, IMHO. Even dog training is based on rewarding good behavior --not on punishing every mistake. Doing the latter will cause fear/distrust instead. This is especially the case when a company wants its employees to "innovate". Innovation requires trying out new things. Trying out new things means that there're going be failed attempts left and right. When people know there're negative consequences for failure, they won't even try because they can't trust that management "has their backs"; they don't have a safe environment. OTOH, management can then turn around and punish them for not "innovating" either. It can be a lose-lose situation, and it's up to you to prevent this from happening to your team.

Point #5: No 2 employees do the same kind of work, even if they're in the same team.

Some employees have a stronger desire to try out new things, such as new technologies/tools to solve problems, or work on problems that no one else has worked on before within the team. (I tend to be like this.) Obviously, that takes more time and effort than those who (perhaps more astutely) choose to remain within their comfort zones. It means that tasks are more complex/time-consuming because employees need to figure out how management's new toys work if they expect to integrate legacy systems to newer ones (This is from a software engineer's PoV.)

The fact that the context and complexity are different for everyone means that you're not really/truly comparing apples-to-apples, especially given that people are rated relatively to their peers. It's easy for a manager to see John Doe as "more valuable" than John Smith after seeing that JD completed 50 tasks but JS only completed 20. The fact that the 50 tasks required less effort than the other 20 larger and more complicated tasks tends to be ignored. Don't let this happen; it's unfair to your team.

Point #6: Trying to rank someone "fairly" by using a process that is, by definition, unfair is an oxymoron.

It's assumed from the outset that there are "poor" performers in the team, regardless of what reality is. It tries to force the reality square peg through an idealistic round hole. If you have some Team-A and some other Team-B, and management cancels the project Team-A was working on, guess which team is going to be seen as the "poor" performers due to no fault of their own? Well, Team-A of course! It's usually reasoned that, since Team-A never "delivered" anything, Team-A must, therefore, not be "productive". (An obvious non-sequitur fallacy.)

I've seen 30+ member strong teams get gutted down to barely 5 geographically distributed individuals over the course of several waves of layoffs in roughly 2 years precisely because of this. (Some months after ratings seems too coincidental to ignore.) In short, those at the bottom end up paying the "broken plates" of those at the top. If you seek fairness, you cannot allow this to happen. The only way to win in a rigged contest is to not play the game.

2. It Expects Pre-Determined Results from the Outset

This is a self-fulfilling prophesy. Top management will look at the bell curve distribution and observe that:

• 10% are above average
• 80% are average
• 10% are poor

Then they make sure their employee rating distribution fits distributions already observed. Then said statistical data is used to justify the team's rating distribution. In short, this is a textbook example of the fallacy of begging the question (i.e. circular reasoning).

As an example, you could create a team of 10 high-achieving individuals, and if you use the stacked ranking system, you've guaranteed that 1 of them will be rated "top performer", 8 will get an average/mediocre rating, and 1 will be rated poorly from the outset, before you've even begun any sort of evaluation.

I've read that Microsoft, Adobe, and even GE (the one who first popularized this rating system), among others, have all abandoned this rating process for the failure that it is. I don't know why other companies still insist on using it...

I've had peers really concerned/afraid of accepting promotions, refuse to transfer to a different team to try new things they're interested in to grow their careers, and so on precisely because they know it increases the odds of being rated as a "loser".

3. It Punishes Real Teamwork

The system encourages the opposite of teamwork.

A company that claims to "value" team-work while only rewarding/punishing individuals is rather inconsistent (read: hypocritical) at best, wouldn't you say?

The issue here is that, under a system that evaluates employees as individuals, there's no reason/incentive to help others in a team --at least from a purely logical/rational PoV. Even companies are careful to avoid cannibalizing their own products...

Can you imagine, if SONY and Microsoft were on the same "team", SONY diverting its own resources to help Microsoft build the XBOX faster and better while SONY itself tries to do the same with the PlayStation? Yeah... not gonna happen.

The reason is simple: Any time that John Doe spends helping John Smith complete his tasks is time that JD is not spending on his own tasks. Since tasks completed by JS will have JS's name on them, and since JD will be evaluated based on the completed tasks he himself had assigned, it's easy to realize that it's not in JD's best interests to help JS when JS gets stuck --even if it would actually help the team and the company as a whole.

As others have already noted, the Vanity Fair article, Microsoft's Lost Decade shows this is more than just some hypothetical scenario. It cost Microsoft the ability to produce a smart phone to complete with Apple early on, which was the outcome of a toxic culture produced by the underlying process. No one wanted to be rated as a "failure" and the only way to achieve that was by working to get the other team to be rated as the "failure" instead; they ended up sabotaging each other.

A tree is known by its fruit. A bad tree cannot produce good fruit just like a good tree cannot produce bad fruit. In short, you need only see the results of applying this rank-and-yank process to the workforce to realize how internally divisive and destructive it actually is.

Think of the Olympics: How can you make sure you win the gold medal? Well, by making sure no one else can, of course! If this were a game, it'd be a free-for-all. If this were a movie, it'd be Highlander, where "There can be only one!".

4. It Undermines Trust

The system is inherently very secretive and lacks transparency. I've never met anyone who was either able or willing to explain it. It also sends inconsistent messages to the workforce.

Yesterday's "top" performers are today's "average". Today's "average" are tomorrow's "poor" performers. Tomorrow's "poor" performers are in danger of getting laid off. Performance measurements don't really "measure" performance. They're basically thinly-veiled witch hunts with the objective of figuring out how to justify a "poor" rating on someone that's very unlikely to actually deserve it and who has been working just as hard as the guy/gal in the cubicle next to you. I've seen management openly, yet politically, admit that upper management has provided "guidance" and "set expectations" about how the process is to work (read: how they want results to look like).

Years ago, when I was a new-hire at a former employer (and before I knew better), I received very positive feedback during my first year, including from management. Then, during ratings, the very management that had been praising me throughout my 1st year later put a "below expectations" rating on me out of the blue. When pressed, my then-manager admitted that they had a "quota" (his word). In short, I was the de facto sacrificial lamb in the team to honor the supreme process, for no reason other than me being "the new guy".

Trying force pre-conceived agendas on reality is not fair to the workforce at any level and makes no sense at rational/logical level.

5. It's Subjective and Arbitrary

The system is based on subjective and arbitrary perceptions rather than objectively measurable criteria.

Is it even possible for a person to come up with a non-arbitrary set of criteria to measure another person? This is not just a hypothetical philosophical issue; it's at the very core of any system of measurement. It can be argued we can come close enough to something that most people would consider reasonable, at least on paper, but I think it breaks down in practice when people try to apply it.

As briefly noted, many people tend to do a lot of necessary behind-the-scenes work, a lot of which goes unnoticed, especially by those who often use it the most.

Consider the StackExchange staff. They keep this infrastructure up and running, but most people will just take that for granted. After all, it's "just another web site in a sea of millions of others sites, so it's expected to be up and running".

Yet that's precisely the point. It's like a special effects team in a high-budget movie. You can only tell the SFX team did an outstanding job when you don't notice that they actually did their job!

Yet, behind-the-scenes work, in several cases I've seen (particularly in test and tools teams) tends to go unrewarded and/or under-appreciated. If you want to try and be fair, you must make sure you take these details into account and represent them well. Unfortunately, it's often the case that managers above you, who are far removed from those in the "trenches", make the decisions and will just overrule your representation with their more authoritative, though not necessarily better-informed, opinions, which goes back to the previous points I made.

Conclusion and Final Thoughts

Ultimately, the rank-and-yank process promotes a dog-eat-dog environment when consistently followed to its logical dead-end. Most (not all) people choose to be illogical and don't do that because they want to be good team players, but the point is that the process itself does not encourage this altruistic behavior/attitude. If it's at all possible for you, try to use your influence to move away from the rank-and-yank system.

If you really want to be fair to your employees, you cannot play the game.

If you're stuck with the process, as you seem to be (at least for the time being), then at least consider the following:

1. Refuse to force data into the pre-conceived expectation "buckets".
2. Refuse to pit employees against one another; instead, represent the work they do and the value they create, regardless of whether you get along with the specific individuals.
3. Realize that if you cannot quantify the value they create, based on your direction, then that's on you and not on them. Take full responsibility for this; we're all human; don't let them pay for your mistakes.
4. Protect your team. Be clear and transparent with them, always.
5. Reward attempts to innovate, even when they don't pan out.
6. Hold yourself accountable for the team's overall motivation/morale, performance, etc.

Avoid playing office politics with your team. I've discovered it's often the case that, due to a fear of retaliation of some sort (e.g. through a ratings process, not surprisingly) employees may not really share feedback they'd actually like to share --and that you need to hear. (Ever wonder why the content of hallway conversations is so different from the comments you hear during meetings?) A work culture where people are expected to "toe the line" will be inherently demotivating; this is based on first hand observation and, sometimes, experience.

Perhaps even suggesting that teams, rather than individuals, get evaluated and rewarded as a single unit might actually help encourage real teamwork (i.e. shared success/failure). That way, even if John Doe spends his time helping out John Smith, Alice, and Bob to complete their work, it's clearly acknowledged that they're all contributing to the team moving forward b/c Smith, Alice, and Bob wouldn't have been able to finish their tasks without Doe's help --or at least not as quickly.

For example, I've seen management treat development teams as "an investment" and look at test and tool teams as "an expense", and yet, the products would've never made it out the door without any one of them having done their part. It's a real shame.

Sun Tzu had several things to say abount management. These few seem potentially relevant:

1. "If orders are unclear, it is the fault of the general. But when the orders are clear, and are not carried out, it is the fault of the officers."
2. "Regard your soldiers as your children, and they will follow you into the deepest valleys. Look on them as your own beloved sons, and they will stand by you even unto death."

I hope this lets you see that you bear more responsibility/accountability for the performance rating of your employees than they do themselves and why I think this is the case. Not only that, what you (don't) do also impacts their morale, team dynamics, and trust, among other things.

Lastly, this article on Compensation, by Mary Poppendieck, looks like a good and relevant read.

References

1. Microsoft's Lost Decade
2. Inside Facebook's 'cult-like' workplace, where dissent is discouraged and employees pretend to be happy all the time

---

If this made you sad and you care for some humor, Dilbert comic strips pretty much sums it up in a few frames:

1. Excellent Work vs Evaluation Excuse
2. Relative Performance Comparisons
3. Above and Beyond
4. Position-based Evaluation
5. Ranked by Performance
6. Predetermined Results
7. Company Policy on Performance
8. Fire the Bottom 10%

Answer 4

You cannot. Stacking system is inherently unfair. Bending the results to fit the theory is hardly ever a good idea.

The individuals should be judged on their own merit. As an above-average developer (or so I'd like to think) should I be penalized for working in a great team? And rewarded for working in poor team - as my relative performance is seen as better?

What if the whole team of four did an excellent job? Would that translate to 1x A, 2x B and 1x C (however the system names the grades)? That's madness!

Not to repeat every point, I recommend a good article:

http://www.poppendieck.com/pdfs/Compensation.pdf

Advice? Abandon the stacking system.

Answer 5

To elaborate on the disadvantages of the stack system over time, I spent a long time in the US Air Force, which uses a similar system in theory. In reality, what ended up happening is that it was career death to get anything below a 5 (the scale was 1-5).

The performance reviews were weighted with regard to your promotions so heavily that it caused supervisors to simply give every airman they didn't absolutely hate a 5. Naturally, people higher up in the chain hated this, since they wanted to identify true top performers, and so tried to increase the requirements to achieve a 5 by adding various volunteer time requirements, office awards, and other things, which again just made the supervisors game the system by parceling out the Airman of the Month awards to each airman in turn, and setting mandatory volunteer events.

Woe to you if you got stuck in a squadron whose commander wanted to "fix the system" by enforcing a strict quota, because by giving even 70% of your workforce a 4 or lower, you were sentencing those people to go up against the majority of competitive airmen who generally were "Firewall Fives". I saw two people kicked out of the USAF for substandard performance--people who would have been fired immediately in the civilian world--and even under such a commander who wanted to enforce a quota, their rating was mostly "3".

Basically, these systems have a well-documented history of causing absolute havoc, rather than their stated goal of identifying top performers. If you absolutely cannot change the system, you need to at least minimize the negative effects. For example:

• They're incredibly easy to game, and thus I would absolutely not publically begin to take things that are tangentially related to work performance into account. You might think that you want to reward the person stocking the soda fridge, but unless you have roles like this that everyone can fit in, you will cause workplace drama over something incredibly trivial the second your employees catch on to this. You don't want your employees suddenly wasting manhours concerning themselves with finding busywork. This applies to awards like Employee of the Month as well--these immediately turn into cesspools of office politics under such a system.

• If you want to have a personal relationship with your employees, such a review system immediately opens up feelings of nepotism. You're going to have to be impersonal if you don't want people to start thinking "Oh, that person's always joking with the boss, of course they're going to get an A." YOU are going to have to be constantly aware of this. In the Air Force, this was a major reason fraternization with higher ranking people was so frowned upon. You cannot have a completely transparent, non-subjective system, and you cause more problems the more you try to quantify every useful thing, so as much as you seem to want to avoid it, you're likely going to have to go the opposite direction and also withdraw from your employees to avoid these issues. End result is that you might not be as popular as you would like, but these systems make it impossible to be the great, well-liked, friendly boss you would like to be while maintaining an appearance of impartiality.

• If you do settle on a decent metric to use, consider not actually telling anyone what it is. You can't quantify every activity that's useful to your office, and you can expect that if what you have quantified is well-known, all the focus will be on those activities and those activities alone, to the detriment of actually accomplishing the goal. I would perhaps rely on input from the people directly under you--have them inform you of specific achievements by each member of their team, write them down in a notebook, and as each performance review is due, look them over. You'll probably find you can break out each into various categories: performance based, morale based, etc. If John writes 200 lines of code a day (an awful metric, I know), and Jill writes 180 but also stocks the soda fridge, then you can decide how much of a performance hit you think Jill "should" take and if her extracurricular activities put her above or below John. Your top performers and lowest performers are generally the easiest to identify--it's ranking the average people that's by far the hardest.

In summary, this is going to be a lot of work for you to get right, and can very easily hurt performance if you don't. If this system has already been in place before you arrived, you'll likely find that the problems I mentioned above are going to already exist in some form or another. Be very careful making sweeping changes to the system if this is the case--you will immediately be the one throwing everyone's career plan into crisis. At the end of the day, I would seriously ask yourself if this job is a good fit for you as well.

Answer 6

I expect that you will find any objective scheme you try will fail for some individuals in the organization. A score based system presumes that all jobs are created equal. And that all components of the scoring system are properly weighted against each other.

If Jane is an idea person, while Bruce is great at talking a difficult client off the cliff, which one is better? Jane may score lower on the customer rating side while Bruce scores lower on the idea generation side. But both add unique value to the organization.

An objective system presumes that a specific rating in each of the categories has been properly weighted in its contribution to the overall score. Is an excellent rating in cross-training the same value to the org as an excellent rating in performance on assigned tasks? What contribution do each make to the final score? Is cross-training half as important? A quarter? Do the same weights work for all teams in an org?

Who is responsible for the decisions?

I posit that attempts to build an objective system are ignoring your responsibility as the manager. A part of your job is to identify, hire and retain key talent. If you let the numbers do it, you are relinquishing your role in seeing beyond the numbers to the value each individual brings to the team. This is the hard part of being a manager. The responsibility for these decisions lies with you, not a system. When it comes to measuring the value of a person, I don't trust a system that can't see beyond what it is measuring. Or one that treats all roles and individuals as interchangeable.

How to be fair in a subjective ranking system?

Fairness boils down to being truthful with yourself as a manager. Which of the staff add the most value to the team and it's ability to delivery on its mission for the company? It's asking the impossible questions when you must choose between who gets a raise and who doesn't, when everyone on the team did a great job.

Start with performance ratings

It's important to start with a shared understanding of how to rate the performance of each member of the various teams. If you don't start with a similar method amongst teams for rating performance or value on some scale it will be very difficult to build a consolidated ranking of multiple teams.

To be fair among a group of managers, the managers need to talk about the impact their respective employees have had on the goals and mission for the organization. What is each person's role in the org and how well are they doing at fulfilling that role? Each manager needs to come prepared to defend their respective staff, both their ratings as well as rankings.

Where this gets really hard is when you have people that have very different roles. How do you compare a group admin to your top engineer and your top sales person. Letting what you believe is an objective system make that decision is not fair to a diverse group. It doesn't acknowledge the different roles each person plays and how it's possible that a group admin could well be the top ranking ranking person at any given time based on how they execute on their role vs. the others.

How to consolidate multiple teams into a master list?

On the surface it may seem fair to rank the top member of each team to fill the first 1-n slots of the collected ranking table and move to the 2nd on each team for the next n slots. This presumes all teams are the same size and each team has the same distribution of high to low performers.

Reality is messy. Team sizes vary and one team may have two superstars while the next has none. If you have a poor performer on a team of 4, should they be ranked higher than someone doing well on a team of 8?

If your final ranking would differ if certain staff are on the same team vs. different teams then your system is not fair. It would effectively be punishing someone for being on the wrong team.

What can help in consolidating a list is for each group manager to assign a performance rating to each member of the team. All team members that are rated as exceptional performers are collectively ranked, and so on down your performance rating system. This acknowledges that members of a specific rating group are not evenly distributed across the teams.

Is it fair?

Ultimately, virtually any ranking system has problems. It's an attempt to quantify the value an individual has to an organization. It's a snap shot in time based on the perceptions of the people involved in doing the rating and ranking.

It will not be fair to everyone. And the answers may be different this week vs. next.

Accept that, do the best job you can and move on. Acknowledge errors you've made in the past and work to do better the next time you are involved in this activity.

Do your best to act fairly, without prejudice or malice.

Answer 7

Personal opinion: I am not convinced this sort of system does more than justify the decisions manglement have already made, and give them an excuse to draw an arbitrary line between people who realistically should rank comparably.

Counsider the old boxer problem.

      BOXER.     A.           B.       C.
  strength       Average.   Worst.    Best
  speed.         Best.      Average.  Worst
  agility.       Worst.     Best.     Average

Using a simple "two out of three wins" model, boxer A beats B due to good strength and excellent speed Boxer B beats C due to good speed and excellent agility. But C beats with good agility and excellent strength. Who is "best" depends on what you need them to do, and there is no absolute ranking of all three.

Real world ranking is often similarly non-transitive. Worse, it is entirely possible for many people to rank equally to any justifiable degree of precision. But that doesn't satisfy the beancounters' need to draw lines at fixed percentages in order to "reward the best, fix or fire the worst."

These systems also lead to grade inflation , where somehow meeting all the requirements is paradoxically not considered doing your job adequately.

Answer 8

we compare A4 with B4, C4 and D4(all are similarly experienced) while A1 is compared with B1, C1 and D1 (all similarly experienced). Then A4's position within A's team is determined, subject to how A1, A2, A3 fared when compared across their peers in teams B, C and D.

What you are describing is the combination of a "level" system with a "stack rank" system, and I can tell you right now, that is astonishingly demotivating. It seems like it ought to be fair; people are compared to their peers, right?

Well, consider this scenario: We have A1, B1, C1, D1 all at "level one", and A1 is clearly the strongest team member. So we put A1 in the top 10% bracket, grant a nice bonus, and make A1 eligible for promotion. Then we promote A1 to level two, and hey, it is review time again, and now our top performer A1 is being compared against all the level two people for the first time, and someone has to be in the bottom 10%; plainly it is A1.

If A1 was any good by level two standards then A1 would already have been level two, right? And we certainly can't say to any of the long-time level two people that A1 is better than them; A1 just got promoted last month! And in our stack ranking system we have to give someone the shaft at level two, so A1 it is.

I have seen this happen over and over again, and you know what it does? It encourages people who have just been promoted to leave the company, that's what it does.

But we don't even have to look at the bad outcomes to see why this idea is terrible.

Suppose you have a machine that produces something valuable. Surely to goodness the right question is "how do we keep the machine functioning efficiently?" But the question that level / stack rank systems seek to answer is "what is a total order of all the parts of the machine?"

Imagine trying to do that for an aircraft; we're going to take several million parts that all make a machine that flies, and rank each part by importance. Then we're going to divide those rankings into completely arbitrary levels, and throw away the parts that are at the bottom of each arbitrary level. Sure, the wings are important, but surely one of them is more important than the other, and we don't need to keep them all around, right? And sure, the peanut storage crates are not strictly speaking vital to the safe operation of the craft, but one of those peanut crates has got to be the best of all the peanut crates. We want to make sure to keep that crate around. Ridiculous.

So how do you fix this? Plainly you don't. But you can keep from making it worse than it already is. Don't implement a level system on top of a stack ranking system. All that does is accelerate the rate at which your top people will leave.

Answer 9

I am stuck in this horrible system too. People can tell you to change the system, but it is very hard to get time from the CEO who is 50 levels above me.

I don't track individual metrics. The reason being that people start to game the metrics and they become inaccurate, they don't tell the full picture and also they take away from doing teamwork.

Tell the team what they will be evaluated on during the year: 1) Quality, Teamwork etc

Then I do quarterly performance reviews where I collect feedback from the entire team. I have leads whos job is to closely monitor the performance of team members. I weight their feedback higher.

I try to rotate people between leads as well during the year. I try to have multiple people work on projects to it is harder to hide bad performance and it makes the peer reviews more accurate.

I am honest that it is a stack ranking system, which hurts morale. You have to be careful that the person with the peer review does not have an agenda aka sinking the person who they feel that they are in closest competition with.

If you could distill knowledge work down to a formula you would not need managers. Unfortunately, it is up to you to decide based on factors you and/or the company values. Being honest about what is valued and how the system works will at least redirect slightly the hatred you will get from the X% who are mandated to not get a exceeds expectation on their performance review.