-3

I’m a software coordinator and management wants a numerical assessment of all employees in the company. I am not a developer myself.

The only numbers I could think of are lines of code and changes (the red and green things) so that is what I plan to use to measure productivity.

example commit review (an example of what I mean by red and green things)

Is it a good metric for employee assessment? What is a good number of lines of code for the purposes of stack ranking?

11
  • 1
    What programming language? This is fraught with issues, but let's start there. Java is going to have far more lines than Python. – Matthew Gaiser Feb 19 '20 at 21:45
  • 3
    I'm voting to close this question as off-topic because it is asking for trivial knowledge available on a quick google search. – nvoigt Feb 20 '20 at 10:50
  • 1
    @aaaaasaysreinstateMonica why not do it yourself? This is a new user, it's a fair and interesting question and this community is screeching because they don't like the premise? I really don't want this site to become like SO, but it seems to be going that way... – bharal Feb 21 '20 at 17:35
  • 1
    @nvoigt i'm also curious as to why, after not being on this site for a yearish?, every second user has a "reinstate monica" name – bharal Feb 21 '20 at 17:37
  • 1
    @bharal That's a long story, you can start here to read up on it. – nvoigt Feb 21 '20 at 17:39

10 Answers 10

22

Measuring the worth of a coder by the amount of code they produce is not a good idea, and there are several reasons why.

  1. Not all code is created equal. Some blocks of code are very simple and can be banged out in minutes, while other blocks of code can be very complicated, taking lots of research, and might require hours or even days to make sure they are written correctly.
  2. Some programming languages are more verbose than others. A piece of code written in Java might be naturally longer than a piece of code written in Python, even if they do the exact same thing. Your plan would automatically value some programming languages over others.
  3. This would create an incentive for your programmers to write mode code, instead of better code, which leads into the most important point...
  4. More code does not automatically equal better code, and in fact the exact opposite is often true. Smaller amounts of code are often easier to read, understand, maintain, test and fix. This is part of what we call "code health". Just because code works, that doesn't mean the code is good.

If you do intend to measure coders by how much code they generate, it's going to pretty quickly backfire on you and it will be bad for the company in the long run.

The underlying question here is "how can I quantitatively measure the quality of an engineer" and the best answer to that is you probably shouldn't. You need to look at more qualitative measures, such as what projects they deliver on, how effectively they document things, how well they implement tests for their code, how well they work with other team members, and things like that.

2
  • 1
    Introducing new features while, in net total, removing lines of code is a true mark of a quality developer – ig-dev Feb 19 '20 at 22:48
  • @ig-dev The flip-side being that any quality developer, given the wrong incentives, should be able to automate the conversion of a nice compact program into an extremely long one... – Geoffrey Brent Feb 19 '20 at 23:34
13

Lines of code is probably the absolute worst metric you could choose to measure either productivity or effectiveness. What ends up happening is you end up with a lot of code bloat because developers will learn the metric and figure out how to write their code to include the maximum number of lines possible.

If you want to measure productivity, team metrics are what you should be using (IMO) based on the features being delivered:

  • Planned vs actual stories committed (are they over or under committing?)
  • Average velocity sprint for sprint (specifically the change in velocity)
  • Code churn
  • Bugs identified per feature

There are a whole host of metrics beyond this that are obviously useful, but the more metrics you measure the more paralyzed you'll become in understanding the narrative of the team. The narrative is probably the most important piece to digesting any specific metric. Some metrics will look really impressive until measured against other metrics when compared to the narrative. I've found teams that will game the metrics and have a very high velocity. This looks terrific until you measure against their bugs returned per feature, and now it just looks like seagulls pecking keyboards.

Note: I've focused on the concept of measuring coders in terms of a team. Teams will naturally suss out low performers when the team begins to work together and start achieving goals against the collection of metrics. If you find that the team doesn't begin to perform or at least steady out their metrics, the problem isn't them... it's you. This is a tough one to swallow that I've personally had to face.

4
  • @JoeStrazzere: absolutely, that's why my focus is on the need for multiple metrics and discovering the actual narrative. If you focus on only one, you'll get a team that does that one thing at the cost of things you're not looking at. My list above is nowhere near exhaustive. – Joel Etherton Feb 20 '20 at 14:18
  • I don't think we're in disagreement. Metrics are only a part of a team narrative on performance. They're only sign posts to potential problems. The narrative is formed through interaction, peer review, and, frankly, produced results. Grading on a metric is foolhardy (good or bad). But ignoring metrics when they can be telling you something is wrong is just as foolish. – Joel Etherton Feb 20 '20 at 17:30
  • @JoeStrazzere: Are you attributing that quote to me or perhaps implying that my answer in some way validates OP's adherence to a misguided choice? I'm not sure what point you're trying to make that I haven't already made. I don't use metrics to measure people. I use them to help me determine the narrative of a team and its true indicators of performance. – Joel Etherton Feb 20 '20 at 17:38
  • Let us continue this discussion in chat. – Joel Etherton Feb 20 '20 at 17:41
8

I had a wise boss once who said "A good software developer can double his rating according to any performance rating, without any increase in productivity".

Google for "Microsoft lost years". Stack ranking turned Microsoft into a less than mediocre software company for many years. Because what mattered to people wasn't doing a good job themselves, but making sure with any amount of backstabbing needed that others are rated lower than yourself.

Stack ranking was in fashion once. Nowadays it is totally debunked as a feasible management method, and only idiots would use stack ranking. Sorry if your poor management is offended by that :-)

1
  • 1
    Exactly - before introducing any metric, you need to ask how it could be abused. – Robin Bennett Feb 20 '20 at 11:01
5

If you want to spot the really good developers, measure how many lines of code they can delete from the repository every day, while still having the code pass every test. Any fool can write lots of lines of code. Removing them is much harder.

If SLOC per day is the measure you use, then people will do everything they can to meet the target. But setting arbitrary targets rarely delivers the desired result. What happens to the poor soul who is given the task of updating the documentation set, and writes no code at all for weeks on end? Are they worse than the person who just writes reams of overly verbose code, just to get their SLOC count up?

2
  • 2
    Documentation? You better define that as a company like this will have never heard of such a thing. – Matthew Gaiser Feb 19 '20 at 21:51
  • 1
    This isn't any better than counting additional lines. It might actually make it worse. Developers would be write massive one liners instead of spacing it out which leads to incredibly complex code being jammed into one line which becomes next to impossible to debug without heavy formatting. – Shadowzee Feb 19 '20 at 22:21
3

management wants a numerical assessment of all employees in the company.

All employees? Fair enough. What is a reasonable number of decisions that a manager should make each day?

0
3

Seems to me as a programmer to be a terrible metric, as it would promote writing bad code just to increase the lines of code you write in a day. For example take a simple loop:

for(i=0;i<=9;i++){
 doSomething(i);
}

Now my productibity is 3, but if I did it the bad way:

 doSomething(0);
 doSomething(1);
 doSomething(2);
 doSomething(3);
 doSomething(4);
 doSomething(5);
 doSomething(7);
 doSomething(8);
 doSomething(9);

I just upped my productivity by 333%! Surely I deserve a big raise now, right? Both examples do exactly the same thing. Meanwhile my co-worker that is hunting down a bug might only change one character in a day, but with that turned the code base from being useless to being usefull and making the company money.

So while lines of code can be a part of the productivity assement, on it's own it's only useful to figure out if the developer is still alive or not (with no lines committed means he is dead, everything else means he's alive and working).

2
  • Or, in a more sensible language, x = [doSomething(z) for z in range(5)], so by moving to Java I've increased my productivity. – JohnHC Feb 20 '20 at 14:31
  • No no no, you reduced you productivity, you just reduced you commited lines from 3 to 1! – user17645 Feb 21 '20 at 10:13
2

What is a good number of lines of code for the purposes of stack ranking?

A fish.

The ONLY value I can think of in this would be to collect the numbers over many developers over many years in the same language then compare the results of developers who were promoted or otherwise deemed stars with those of the ones who weren’t and see if any patterns have emerged. From code being too dense to be understandable to poor practices like copy and paste vs. refactoring and correct decomposition of problems you can have so many different coding decisions alone contributing to too many or too few lines of code. Next, your seniors and leads may be writing less because they’re mentoring, reviewing or being thoughtful and producing 30 lines that always work vs. 200 another may produce only to get fixed every three months.

I understand them wanting to understand what’s going on “under” them and even the thought process that might lead to this. But counting lines of code would be as effective as judging them on the color of car their dad drives.

0

I'm a programmer and this is actually a very hard question and it depends on the complexity of the problem. If I'm doing simple stuff I can easily do over 100 lines in an hour, but if I'm working on something difficult the research and logic will slow me down enough to potentially get in only a couple hundred lines in a whole day, if you take into account having to test your code as you go, which means erasing lines and rewriting or commenting out code and retrying. That's probably a bad metric, but honestly using something like simple features added might be a better approach. That would probably involve more overhead and testing to keep track of.

https://www.cprime.com/resources/what-is-agile-what-is-scrum/

This goes more into more details on methodology to help with this kind of environment. Typically you have the developers vote on how many points they think a task is worth, based on complexity, size, etc. and you only assign a certain number of points per sprint(2 weeks to 1 month timeframe). It still has it's issues and it's still not perfect, but it tracks raw performance pretty well. Not the greatest for tracking progress on a project due to bugs (unintended effects), defects (code that doesn't follow guidelines, etc.), and scope creep (growth of a task based on factors not considered or work not accounted for).

5
  • I don’t understand that stuff, but I assume that one block of code takes about the same time to write as another? – rankmycoders Feb 19 '20 at 21:59
  • 2
    @rankmycoders, the number of lines and the time it takes to write them are directly affected by code complexity. Remember programming languages are in fact just languages. By comparison, does it take the same amount of time to write a 5 page research paper as to make a 5 page list of movies, songs, and books you like? – LightBender Feb 19 '20 at 22:02
  • 2
    @rankmycoders that is definitely not a valid assumption to make. Depending on complexity, some blocks of code can take far longer to write than others. – TheSoundDefense Feb 19 '20 at 22:02
  • @rankmycoders depends on length I have personally worked on blocs of 700 lines or so (Fortran 77) and I have seen SQL blocks of several K (ok it was the largest IBM system in the world) at that time :-). – Neuromancer Feb 19 '20 at 22:08
  • @JoeStrazzere For more money, I could write a program that stretches out the 100 lines of code to 10.000 lines of code, still doing the same thing. If I lend that program to my colleagues, we could be super programmers m-) – nvoigt Feb 20 '20 at 14:20
-1

Everyone has noted that "number of lines" is a bad idea. But, at the same time, management wanting an assessment of employees isn't a bad idea. How else do they know who is performing well, which divisions have better talent and so on.

It is true that "someone will try to game the system". But that isn't actually a reason to not have a system. It is a reason to make sure that the system is consistently vetted and it's outcomes measured against the company goals (one of which is probably "make a profit"), and so maybe over time the system needs to be changed. But you still need a system.

So, given you want a system, how do you work out if it is the right system? You have to realise, before you embark on this, that whatever system you use will produce results, so you have to think "what result do I want".

Once you've worked out that, then you have to work out how to measure the result that you want. And once you've worked that out, you can measure how well your system is performing. At this stage, you will know when/if you need to make changes to the system or not (and if those changes are "good" or not).


So, what are you trying to produce? I mean, you need to make a report, but what is that report going to be used for?

Knowing who is the best, say, HTML developer isn't a really useful piece of info to have. After all, what's the point for management to know this? They can hardly make this person the head of engineering - or a sales person. They just know this person is great at HTML - are they a leader? Are their ideas good? Just knowing "best HTML developer" just means that that person should do HTML - which is probably what they were doing before, so it really doesn't help anyone.

So what you might want to enquire then is why they want to know this. Knowing this will help you work out how to grade the engineers. For example, is it because they're working out bonuses? Then easy, you just find out from all the engineering managers which are their best/ok/worse engineers , and a then you can lump them all into buckets and say "these are all As" and so on, because it's not important as to the intrinsic thing you're measuring, but just that you keep the "best" engineers happy.

That the managers might not know or be playing favourites isn't really the point - because the point is to divvy up the bonus in the best way. And if you start losing engineers that the managers wanted to keep because you divvy'd up the bonus in a different way, then you'll be responsible for explaining why you chose some other metric here, and it won't matter because managers will be screaming that your metric sucks and now they are running late on whatever.

On the other hand, it might be some other idea - maybe they're trying to find the smartest or the best people and try to lift them up for promotion. Then you might do two things - rate your engineers on leadership qualities, or rate them on eagerness to learn.

The first one is pretty nebulous, you just ask the managers to rate their engineers based on leadership skills. You can probably find a book or a website that lists some qualities and how to measure them and then give that to the managers and wait for them to give the reports back. Pretty easy.

The second one is more interesting - if your company offers that training stuff, then you can let it be known that engineers will be rated on what they're learning, and how much, and I'm sure that stuff is kind of measureable. I mean, online training things have assessments and give reports etc etc.

But you'll always have problems with things like this in that it might detract from the company goal (ie at the extreme engineers are learning 40 hours a week and not doing anything), so you've got to consider how to mitigate that (limit learning to some amount a week?), or mix it with something else (lines of code?) or the number of jira tickets that are filled or whatever.


But at the end of the day, you need to work out what you want the result to be of all this, and then devise a system that encourages that result. Given we know very little about you and your workplace, that's really hard to make an answer for.

1
  • 1
    This actually has a lot of really solid points. Management does need to know how productive an employee is and knowing that information benefits the company as well as the employee. Also lists a lot of things to consider, like the reasoning which might make a huge difference in the best way to measure the metric. – Shirohige6969 Feb 21 '20 at 15:04
-6

What is a good number of lines of code for the purposes of stack ranking?

You need simple statistics.

  • Ask developers for a trial for a week. Measure the number of lines they commit per day. You will need to make sure your experiment is random (google random sampling if you don't know). Your sample size shouldn't be too small, more than 10?
  • Measure the number of lines committed per day for each individual.
  • After the week, plot their performance on a 2D graph. Fit the curve either with linear regression if linear effects or something like LOESS. The y-axis will be the number of lines, and the x-axis will be the salary (or experience in years).
  • Form a confidence interval around your fitted curve, let's say 95%. Bootstrap the interval if necessary.
  • Present your fitted model to the team. Make it clear anyone who falls outside the 95% confidence interval will draw attention from the management, such as warning letter.

Obvious advantage is that you will then able to use your fitted model as an evaluation tool on any developer in your organisation. Known as predictive modelling, simple application of machine learning in data science.

The management will be able to adjust the confidence interval to whatever they feel necessary. For example, a tight schedule may require a narrow interval than usual days.

7
  • 7
    This is trivially wrong. The base assumption that "more code typed" = "improved product functionality" is fatally flawed and no mathematics will be able to resurrect it. – nvoigt Feb 20 '20 at 14:16
  • @nvoigt This is a Q&A site, not discussion forum. Please follow our FAQ guidline. It's not our job to speculate. We answer whatever being asked, we don't guess. If you require assistance to how to use the site, please raise a question on meta. You have been assuming something not stated in the question. – QuantFinance Feb 20 '20 at 14:20
  • 3
    I do not require assistance, this site is working great, thank you. – nvoigt Feb 20 '20 at 14:28
  • 1
    @dustytrash No there wasn't and yes, I agree. – nvoigt Feb 20 '20 at 16:28
  • 1
    @HelloWorld This is still arguing that this is a good metric. Sure it's not our job to speculate, but if someone asked you how to theoretically blow up a city are you going to tell them it's a bad idea for reason x, y, and z or are you going start giving them ingredients. I feel it's more productive to sway people away from bad ideas than to let their lack of background make their situation worse. – Shirohige6969 Feb 21 '20 at 15:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .