I understand the motivation to weed out non-programmers from programmers. It seems like a lot of blogs either conclude or insinuate that candidates who fail fizzbuzz questions don’t make good developers. Of course, the term “failing” encompasses a number of scenarios:
- Fumbled logic i.e not testing if i %15 == 0 before the other two conditions.
- The loop not counting from 0 through 100 i.e a classical
for (int i = 0; i < 100; i++)
- Using the wrong operator i.e using / instead of % in one of the conditions
- No solution at all (Candidate refuses to write any code)
Of course, the fourth scenario is a big NO but if you were an interviewer and a candidate submits code on paper under scenarios 1, 2 or 3 and you don’t offer them an opportunity to walk you through the code to understand how they arrived at a particular solution or redeem themselves from a silly mistake they can easily rectify, would you conclude that they failed the test?
Would you fail someone who has correctly formulated a solution to a problem situation but fails to write on paper code that would give the desired output on the first run? Does it sound unproductive if the code doesn’t give the desired output on the first run, and is it a realistic expectation at all to always produce code that gives a desired output on the first run when creating software?
Would you hire a developer with a good reference of 2 years of experience working in XYZ technology who submits code under either 1,2 or 3 scenarios? Of course, you would hire a developer with the same credentials who passed the test, but would you base the entire hiring process on whether someone gets the fizzbuzz challenge right or wrong on paper? And while we are at it, would you conclude that the former is better than the latter?
Does the fizzbuzz challenge factor in the fact if people are capable of making stupid mistakes even in the comfortable environment of a text editor they could fare worse on a whiteboard or sheet of paper? And If you are judging someone on the basis on those mistakes instead of the overall thought process, testing, coding style, readability, debugging, understanding requirements, etc. how effective is the fizzbuzz challenge in determining a good hire?
Or perhaps, the fizzbuzz challenge could as well be a misunderstood test after all candidates make mistakes on a dry erase board under interview pressure and sometimes get it right or wrong depending their mind set. In that respect would you agree to the fact that it’s just a test to see who has a clue vs who couldn’t program their way out of a wet paper bag?
Is there evidence showing that candidates who can write the fizzbuzz challenges turn out to be productive programmers and those who don’t are not productive at programming whatsoever? Evidence provided thus far has shown that a small number of candidates can whip out fizzbuzz under five minutes and the rest cannot (Are we talking about a solution that produces desired output on the first run or a decent solution that shows competence?). IMHO it does seem that fizzbuzz checks to see if people can reason through a problem and break it into components. The skills are essential for effective programming, but this doesn’t verify the connection between good programmers and fizzbuzz performance on paper?
I'm not looking for opinions, just facts. If you don't have numbers to back up your reasons keep your opinions to yourself, please. Thank you.
tl;dr - Is there any kind of test such that if administered during an interview process can give an interviewer some perspective into how (good|effective|productive) a candidate is for a developer role?