So is there any scientific research on whiteboard interview effectiveness? There is a tough problem of measuring person's effectiveness, but anything with actual numbers, statistics and conclusions would be great.
How do you measure the effectiveness of an interview?
- Employer happiness with employee who took an offer?
- Likelihood that an employee will accept an offer after having done a whiteboard interview?
- Percentage of applicants who pass the whiteboard exercise?
I know you mentioned the difficulty of evaluation effectiveness, but you then still ask the same question, which seems weird. If you can't objectively evaluate it, then you can't conclusively answer that question.
Maybe some big company calculated true positive rate of people, who passed whiteboard interview successfully and actually do great work.
The assumption that this is a correct metric, is the exact same flawed assumption that leads to thinking whiteboard interviews are a good marker for aptitude.
The issue doesn't lie with hiring people who pass the whiteboard interview (and may or may not be appropriate for the job position). The issue lies with not hiring people who are appropriate for the job position but simply fail to pass a test that does not test their relevant aptitude.
Think of it this way:
An interviewer refuses to hire anyone who spent less than five years in college. The goal of the interviewer is clear: weeding out the "unskilled" applicants, so that only "skilled" applicants are evaluated. However, his metric (time spent in college) is flawed.
I use "skilled" and "unskilled" as an oversimplified discriminator to the interviewer's opinion of the needed skills for the position. I'm not trying to label people.
- Two types of people will pass the test:
- Those who achieved a master's degree (skilled)
- Those who had to often redo courses and eventually managed to get their bachelor degree, almost by brute force and sufficient funds for tuition (unskilled)
- Two types of people will fail the test:
- Those who quit studying halfway through (unskilled)
- Those who were so smart they completed their curriculum in half the time (skilled).
This is why the metric is flawed: it doesn't actually separate the "skilled" from the "unskilled". It does not actually achieve the goal that it is implement to achieve.
The issue with whiteboard interviews is the same. It is used to weed out people who lack certain skills, but it tests the wrong metric and therefore leads interviewers to dismiss applicants with the appropriate skill, as well as potentially offering jobs to applicants based on the wrong skill.
To sum it up in a quote (that is often wrongly attributed to Einstein):
If you judge a fish by its ability to climb a tree, it will live its whole life believing that it is stupid.
The big unmentioned factor here is what the interviewer expects to see on the whiteboard.
- If they expect to see code with perfect syntax; that's simply an unreal expectation.
- If they expect to see a basic visual representation of e.g. how a linked list is different from an array; that is something that can meaningfully be explained using a whiteboard.
The main reason seems to be, that people think, that result of whiteboard interview doesn't resemble work environment.
I, for example, am really bad with remembering syntax by heart. However, I am really quick at googling related topics, finding an example snippet, and tailoring it to my needs. I know this is tooting my own horn, but I can outperform most colleagues in terms of speed of implementing something from scratch.
However, if you put me in front of a whiteboard and ask me to write the code for deserializing an XML file, then I wouldn't even know the class used for (de)serialization.
This may lead interviewers to conclude that I don't even know the basics and my coding repertoire is too small; whereas the opposite can be true as well: my coding repertoire is too big to remember all syntax by heart, and I know more than just the basics to a point of simply not focusing on the basics.
This is why "not resembling the work environment" defeats the value of the whiteboard exercise. Not everyone works the same way. Reciting knowledge by heart is not the primary metric by which you measure a developer's aptitude. And that is the crux of why the whiteboard interview is not a good approach.
If anything, you should be measuring a developer's ability to adapt to new systems and to understand existing code, as that is a skill that is much more valuable.
I was given this type of interview once. They had printed out their derived
DbContext implementation, and asked me to point out what they implemented. They were looking for answers like soft deleting, pagination, auditing fields, change tracking. They had also introduced two bugs. My ability to evaluate the existing code is much more meaningful than being able to create my own class out of thin air.
Secondly, it leaves the door open to a logical inversion fallacy.
Let's say the interviewer asks you to recite all the methods of Entity Framework by heart, ordered by length of method name (I know it's a silly example, but I want to use an example where we don't get distracted by arguing what the best answer is).
If the applicant is able to do this, then that is (indirect) proof that he must have a lot of experience with Entity Framework. Without Entity Framework experience, he wouldn't be able to do that.
However, it's likely that the interviewer will wrongly invert that logic. If the application is not able to do this, that doesn't mean that he doesn't have any experience with Entity Framework. This inversion is a logical fallacy.
This is why these question are bad. When the applicant is faced with a question like this, he is stuck between a rock and a hard place:
- The applicant doesn't know if the interviewer has fallen prey to the logical inversion fallacy.
- If the applicant speaks up about the fallacy, and the interviewer disagrees, the applicant risks hurting his chances of getting an offer.
- If the applicant speaks up about the fallacy, and the interviewer wasn't falling prey to the fallacy, the applicant could have hurt his chances because he assumed the interviewer was making a mistake when he wasn't.
- If the applicant therefore assumes that it's better to not being up the fallacy, and he simply fails to do the requested task (which is not a sign of being a bad programmer), then he risks the interviewer assuming the applicant isn't skilled enough for the position.
Regardless of whether the interviewer falls into the fallacy trap or not, the existence of the question always puts the applicant in an awkward position of having to evaluate the interviewer's knowledge (about the fallacy) before he can give an appropriate answer. That's effectively a gamble, and you really shouldn't (not) hire people based on (not) having guessed correctly.