Implementing Peer Review System

Question

I manage a small team of developers at my startup company. I want to evaluate and reward my team and I am thinking about having a peer review system where they evaluate each other. I think it will be fun, competitive and engaging.

For example, we follow an issue tracking system. I am thinking a team member will go through each issue and evaluate the assignee based on performance, e.g. how complex the issue was, how quickly it was solved, was it bug-free solution, how innovative and clever the solution was etc.

Have you seen a peer review system at work? How are they implemented?

EDIT

I went through the answers, all of them nice and thoughtful so thank you. I am aware of the fact that putting too much emphasis on performance appraisal system may actually break it. My purpose of introducing peer review is not necessarily adjusting compensation/increment. Rather it is for adjusting bonuses. Also as a manager I am not looking to get my rear end kissed :) I'm actually looking to automate this process if possible.

Probably should've mentioned in my question first time, I recently went through the Valve Employee Handbook(pdf) and read about their annual peer review system (on chapter 3) and stack-ranking. Valve is doing quite awesome imho, I want to know how practical this is, and implement something like this in my company.

It is my company after all. Perfect place to experiment things like this :)

Answer 1

Honestly, I think that this is a really bad idea. The first time someone disagrees with the peer evaluation they've been given, what happens? Each of your developers are going to have different skills and perspectives to bring to the table and a simple problem for one may be more difficult for another.

I like to try and foster teamwork over competition. A team of developers, in my experience, would rather work together to create something awesome versus working alone to game an arbitrary review system.

If you would like to encourage peer based feedback, I recommend 2 things - 1) Peer Code Reviews and 2) As the manager, come review time, ask the other team members to provide feedback to you, in confidence, about the employee being reviewed.

Answer 2

The closest system I've had experience with was:

1. All employees were required to complete a self-assessment where they rated their skill levels and accomplishments and so on.
2. Managers reviewed employee self-assessments, and added their own feedback, notes, and review items.
3. Rewards were issued to employees with positive assessments, typically in the form of bonuses.
4. Employees could also nominate other employees for rewards at any time (you have to be able to justify the nomination, of course), and each quarter selected nominees (personally I suspect that every nominated employee would be "selected") would be honored with additional accolades and rewards (typically a plaque and an amazon gift card for $100 or so).

Personally I felt that the review process was a fairly substantial waste of time, even if it was good for about $2500 in bonuses each quarter. I'd rather be doing useful things than writing up an assessment about what useful things I do.

Anyhow, I'd caution against putting too much weight on the peer-review system, in terms of the rewards you assign to it. Make the rewards too substantial and you might encourage either infighting or dishonesty (or both) as nobody wants to feel like they missed out on a prize because someone gave them an unfair peer review, and similarly no one wants to be blamed for a friend/co-worker missing out on their reward because they didn't write a positive enough review.

My advice would be to support peer review through things like automated code-review e-mails that are generated by your version control system every time a change is checked in (this has been standard practice at every company I've worked at, and it catches a surprising amount of bugs), and building a simple review process into your ticketing/issue tracking system if possible. Then let people use these tools how they like, without attaching formal to it with respect to expectations or rewards.

Of course you can reward anyone that you notice getting consistently positive reviews, but I'd advise against integrating that as a formal part of the process. Just give people the tools, and let them work out how best to use them.

Answer 3

I can think of a lot of words to describe peer review but fun is most certainly not one of them. If you want your workplace to become a Darwinian hot-bed of survival of the fittest, go with peer review. People will rip each other to shreds to preserve their own place in the hierarchy. Worse, the worst employees (but the most political) are the ones who will be believed over eveyone else. Peer review is a cancer in an organization.

Now code review (evaluate the code not the people) and peers being able to nominate people for awards, those things work fine, but specifically evaluating the performance of the people you are in competition with for raises? That is the bound to end badly.

Answer 4

Adding another now that you've updated...

If it's your company, as in you are the only stock holder, and/or you & your fellow company founders are all in agreement - then what the heck. Go for it.

Reading Valve's handbook, I think starting with a peer reveiw process is putting the cart before the horse. They seem to have gone to some lengths to make a truly flat organization while minimizes or eliminates the legal responsibility of technical management. I suspect that this means the founders have also given up control for company guidance, development, and profit -- they are literally no longer in charge.

It's a different set of cultural rules than traditional hierarchies, and I'm not sure someone looking at such a group from the position of corporate America can make an adequate comparison (that means me...) but I do think that it's a whole-culture shift, and that you probably can't get the value of Valve's overall culture without streamlining all your processes in a consistent direction - so if you have only the peer review part of the process in play, you are likely to experience the grief that comes from conflicting priorities.

For example - if my boss told me that my salary, promotion and raise was his domain but that my 10% incentive bonus was in the hands of my fellow team members - I'd get differently pushy about some aspects of the job in terms of pleasing them and fighting with my boss... this may or may not be a win depending on the team. I could also see a lot of team angst if the team doesn't buy into the process that is driving the incentive, both the process of the work being evaulated (ie - dislike of issue tracking process) or by which evaluations are made and submitted.

It'll definitely shift your culture... the question is, what are the side effects? Hard to know without knowing the group.

Answer 5

OK, I think you have two laudable goals, but that combining them may not be the way to go. But having any or all of the following in place can be a really motivating, or at least culturally useful thing to have in an organization...

1 - Peer Review for Quality

When I first read the subject, I thought "peer review" as in traditional code review - a person works a particularly critical technical area, other people review and give feedback the feedback is addressed with some level of rigor before the work is considered "finished". Peers look for various flaws - sometimes general, sometimes a particular checklist - ideally both.

I find these tremendously useful and almost a must have in a high risk area where a lack of quality will lead to drastically bad results. This is the same idea as having a pal read a paper before you submit it -- you can live too much inside your own head and a second set of eyes can radically improve that problem.

When I first saw "Peer Review" this is what I thought you meant. They can be applied to more than just code - reading through a problem resolution can be just as useful, if there is something to be learned and improved from the insightful critique. The key is that both peers are working to the same goal - better quality for the product or service.

2 - Peer Appreciation

Absolutely there is something to be said for having succeeded in the eyes of your peers. Having a way to have team members give each other a more formal "atta boy" (cause you don't really need a system for people to informally praise each other) - is a great thing. It gives people the power of reinforcement, it's a really great morale boost, and it doesn't take much cash.

My favorites were cases where peers gave each other tokens (freely requestable) and some number of tokens could be cashed in for gift cards and such. It abstracted the money a tad bit, but still put a value on the tokens. And by controlling the token to stuff ratio, the company could throttle the gifts if they get out of control. Usually there is a modest form of paperwork - like you have to write down the giver, the receiver and what was so cool about the receiver's work. That means that people have just a little bit of a demotivator from giving tokens just because of common courtesy.

3 - "360 degree review"

A form of feedback that goes the whole way around (thus the name) - the idea being that the sum total of a good job well done is not just pleasing your boss, but being good with your boss, your peers and any subordinates. This site did the best I've seen at not just selling the idea, but giving tips and tricks.

I haven't done such a thing - I'd like to, and probably will in my current company. But I participated in giving feedback for a boss at one point. No big change there, but none required - the guy was a great boss, I really liked working for him, I said so. So I'm rather glad he didn't change!!

I strongly suggest reading the guidance on this one - they make some really good points in the down side section and then recommend ways to work around the down side as best as can be done.

Summary

Two things - the focus and the mechanism - are important to figure out here. the three examples I gave have three purposes - one is quality work, one is a way for people to reward each other, one is a way to provide meaningful feedback as a way to improve job performance. All three are good goals, all three are important, all three can be implemented with great success. But if you implement without knowing your driving goal, you are likely to create a system that does no good, and potentailly does much harm. From reading your question, I don't get a sense that you know - it could be to improve issue tracking or issue handling. It could be to give others a chance to assess their peers. It could be a way of assessing job performance.

Mechanism - particularly when it comes to feedback for job performance as it relates to incentive payouts and employee promotions - you have to suck it up and be the boss. Having guided feedback as a way to motivate and improve employees has some real merit, but there is a rubber meets the road aspect when it comes to money.

In particular - rewarding work with (big) money is not about pleasing your coworkers or doing a good job in the estimates of your peers - it's also about aiding the business in reaching it's goals in such a way that more pay or a promotion is warranted. I know that comes down to sounding like a lot of hot air - but the thing is - if a guy is helping the business grow or maintain itself - he shoudl be paid more than the guy who isn't doing as much... if we don't find a way (as managers) to incentivize people in this direction, we will end up with a failing business. So the final judgement here has to be a managerial one. Employees do not hold the same responsibility, nor are they accountable for budget.

So if there was one thing to avoid like the plague it would be forming a direct, unambiguous connection between peer reviews and payment/promotion.

As a manager, do I take into account an employees impact on his peers? Absolutely. Do I ask them about their relationships and guage the level of respect & complaints I get response knowing what I know about the work and the personalities involved? Absolutely. Do I have a spreadsheet or other connecting point between peer feedback and salaries? No way, never ever in a million years. Cause there's too many other factors.

Answer 6

Valve could be a good company inspite of this. I think Microsoft has the same ranking system. Maybe their employees are willing to put up with this because of the prestige and level of salary they get working for these two companies.

Many people like working for smaller companies because they don't do these types of things. Everyone on your team knows who is the best. If you upset your best programmer and they leave, you lose a huge asset. One programmer leaving Valve won't break the company and they stand a better chance of replacing them anyway.

It's difficult to give an objective evaluation to someone knowing it will impact my bonus. It is a competition after all. I can't imagine you'll get any positive feedback on this from your team.