we compare A4 with B4, C4 and D4(all are similarly experienced) while A1 is compared with B1, C1 and D1 (all similarly experienced). Then A4's position within A's team is determined, subject to how A1, A2, A3 fared when compared across their peers in teams B, C and D.

What you are describing is the combination of a "level" system with a "stack rank" system, and I can tell you right now, that is astonishingly demotivating. It seems like it ought to be fair; people are compared to their peers, right?

Well, consider this scenario: We have A1, B1, C1, D1 all at "level one", and A1 is clearly the strongest team member. So we put them in the top 10% bracket, give them a nice bonus, and make them eligible for promotion. Then we promote them and hey, it is review time again, and now our top performer A1 is being compared against all the level 2 people for the first time, and someone has to be in the bottom 10%; plainly it is A1. If A1 was any good, then A1 would already have been level 2, right? We certainly can't say to any of the long-time level 2 people that A1 is better than them; A1 just got promoted last month! And in our stack ranking system we have to give someone the shaft at level 2, so A1 it is.

I have seen this happen over and over again, and you know what it does? It encourages people who have just been promoted to leave the company, that's what it does.

But we don't even have to look at the bad outcomes to see why this idea is terrible.

Suppose you have a machine that produces something valuable. Surely to goodness the right question is "how do we keep the machine functioning efficiently?" But the question that level / stack rank systems seek to answer is "what is a total order of all the parts of the machine?"

Imagine trying to do that for an aircraft; we're going to take several million parts that all make a machine that flies, and rank each part by importance. Then we're going to divide those rankings into completely arbitrary levels, and throw away the parts that are at the bottom of each arbitrary level. Sure, the wings are important, but surely one of them is more important than the other, and we don't need to keep them all around, right?

So how do you fix this? Plainly you don't. But you can keep from making it worse than it already is. Don't implement a level system on top of a stack ranking system. All that does is accelerate the rate at which your top people will leave.

we compare A4 with B4, C4 and D4(all are similarly experienced) while A1 is compared with B1, C1 and D1 (all similarly experienced). Then A4's position within A's team is determined, subject to how A1, A2, A3 fared when compared across their peers in teams B, C and D.

What you are describing is the combination of a "level" system with a "stack rank" system, and I can tell you right now, that is astonishingly demotivating. It seems like it ought to be fair; people are compared to their peers, right?

Well, consider this scenario: We have A1, B1, C1, D1 all at "level one", and A1 is clearly the strongest team member. So we put A1 in the top 10% bracket, grant a nice bonus, and make A1 eligible for promotion. Then we promote A1 to level two, and hey, it is review time again, and now our top performer A1 is being compared against all the level two people for the first time, and someone has to be in the bottom 10%; plainly it is A1.

If A1 was any good by level two standards then A1 would already have been level two, right? And we certainly can't say to any of the long-time level two people that A1 is better than them; A1 just got promoted last month! And in our stack ranking system we have to give someone the shaft at level two, so A1 it is.

I have seen this happen over and over again, and you know what it does? It encourages people who have just been promoted to leave the company, that's what it does.

But we don't even have to look at the bad outcomes to see why this idea is terrible.

Suppose you have a machine that produces something valuable. Surely to goodness the right question is "how do we keep the machine functioning efficiently?" But the question that level / stack rank systems seek to answer is "what is a total order of all the parts of the machine?"

Imagine trying to do that for an aircraft; we're going to take several million parts that all make a machine that flies, and rank each part by importance. Then we're going to divide those rankings into completely arbitrary levels, and throw away the parts that are at the bottom of each arbitrary level. Sure, the wings are important, but surely one of them is more important than the other, and we don't need to keep them all around, right? And sure, the peanut storage crates are not strictly speaking vital to the safe operation of the craft, but one of those peanut crates has got to be the best of all the peanut crates. We want to make sure to keep that crate around. Ridiculous.

So how do you fix this? Plainly you don't. But you can keep from making it worse than it already is. Don't implement a level system on top of a stack ranking system. All that does is accelerate the rate at which your top people will leave.