32

In a 2013 NYT interview, some Google guy said:

On the hiring side, we found that brainteasers are a complete waste of time. How many golf balls can you fit into an airplane? How many gas stations in Manhattan? A complete waste of time. They don’t predict anything. They serve primarily to make the interviewer feel smart.

It seems that many companies still continue asking such brainteaser questions, so is the above quote correct? What research is there on this matter?

Should companies continue this practice?

Edit: I'm hoping to get more general answers, and not just for interviews for programming or tech jobs. E.g. I hear that i-banks like Goldman Sachs like these puzzles too. So do some college admissions interviewers. Do they work?

||improve this question
  • 7
    These questions are designed to show if an applicant has skills in critical thinking and abstract thought. It's supposed to reveal if an applicant can realize the pertinent variables and come up with a reasonable algorithm to approach the problem with. It's not about getting the "correct" answer. The problem is while applicants "pass" this test, managers "fail" because the point of this exercise has been lost. – Wesley Long Nov 22 '14 at 14:40
  • 1
    I think this is part of Google's internal research. The discovered there was more consistency in some of their managers to pick candidates rather than all the different tests they gave. Not sure how rigorous the science is behind this "study" but you have to think they're pretty good with the numbers. – user8365 Nov 22 '14 at 18:10
  • Remember that many companies do a piss-poor job of actually training their interviewers. (I'm very aware of this because IBM actually does have a good set of training materials, but despite sometimes being forced to do technical interviews I can't access most of that because I'm not a manager. Grr.) – keshlam Nov 22 '14 at 21:58
  • 1
    These questions are designed to show if an applicant has the patience to put up with mindless requests that this company likely will inflict upon the candidate upon being hired. I like when companies ask questions like this. Makes it really easy for me to decide I don't want to work there. :) – DA. Nov 23 '14 at 6:54
  • 2
    Well, if it was a Google employer asking such a question I'd definitely answer: "Wait 30 second, I'll just google-that-for-you". Really, this is the optimal algorithm to start solving a new problem: search about it. – Bakuriu Nov 23 '14 at 15:48
44

It seems that many companies still continue asking such brainteaser questions, so is the above quote correct? What research is there on this matter?

I examined this practice years ago. There has never been any research which indicates that these sorts of brainteaser questions can predict performance on the job. (Please - someone post reputable research results which prove me wrong! I could never find any.)

This practice is like many in the corporate world - a fad, popularized in some large companies (Microsoft famously), then followed unquestioningly by others who decided to follow the popular fad, or what some considered a "best practice".

I worked at one company where one of the principal developers used to ask candidates how they would go about finding a needle in a haystack. He claimed it "gave insight into how the candidate thought". When I questioned him a bit it turned out that he was asked this question during an interview once and thought it was clever - thus he decided to be clever and repeat it. When pressed, he admitted that it gave him no real insight. And of course he had no data that indicated it had any predictive value.

Many interviewers will rationalize their approach to interviewing. "It's for testing the candidate under stress conditions." "It's for insight into how they think." "It tells me if they are good at approaching ambiguity." But I agree with Google's conclusion - the actual primary purpose is to make the interviewer feel smart. It has no predictive value.

||improve this answer
  • 27
    Out of interest, did you find solid research as to whether any interview questions, of any kind, predict performance on the job? – Steve Jessop Nov 23 '14 at 1:22
  • 2
    +1. I think partly companies ask these questions so they can be convince themselves that they have done due diligence by following an industry standard (as you say), and to make everyone at the company feel like they have a more elite filtering process. – Mark Rogers Nov 23 '14 at 6:00
  • 1
    @SteveJessop that's actually a good question! I can't say I've ever interviewed anyone myself and thought about productivity specifically. It was usually more along the lines of "Are they competent? Do I like them? Will they fit in?" and just assume that they'll be productive. – DA. Nov 23 '14 at 6:58
  • 14
    How to find a needle in the haystack: Step 1, Ignite the haystack. Step 2, Sweep a strong magnet over the ashes. Step 3, I get the job? – user25891 Nov 23 '14 at 18:01
  • 2
    @teego1967: if it is an inherently soft skill, then there might well be questions that appear completely stupid because in the hands of someone who knows how to interpret the candidate's response, they can be used to find good candidates (or more likely, can be used to rule out a subset of the bad ones), whereas for anyone else to ask the same question is sheer cargo-cult :-) I'm not defending these questions, but that's certainly consistent with "it's not whether you get the right answer, it's how you approach the question". – Steve Jessop Nov 24 '14 at 17:41
12

The whole point behind this type of exercise is that they wanted developers who are good at estimating how much in terms of resources it takes to perform any task - You probably always want to be estimating how long a task will take because you don't want to be caught trying to do something in a way that take 10000 as much time as some other procedure.

Having said that, Google dumped this kind of question for interviews. Most likely because this kind of question is no predictor of performance on critical tasks like working out how to implement the specifics of virtualization as a way to get systems quickly on their feet and in particular, implementing the coding necessary to implement systems reliability. Google is VERY data driven and if Google is dumping it, Google is not saying so casually.

Google is probably better off asking the kind of questions that probe how a candidate approaches and solves problems rather than (unwittingly) play puzzle games. They want problem solvers not puzzle addicts. And that's to their credit.

||improve this answer
3

Well, yes and no. If you go for a management consulting job as a junior/no management consulting experience, then this is a nice way to gauge your thought process. Also, to tell how well you've prepared for the interview. It won't quite be "golf balls in an airplane", more "How many passengers fly through JFK in a calendar year?"

It can also be used as a stress test - how fast can you think on your feet, are you capable of coming up with something coherent on the spot, or do you struggle?

While Google might not do the particular brand of puzzle question indicated, it is hardly accurate to say they don't do those questions at all. Google programming questions are just as "unfair" as any puzzler question you might be asked. As ever, prepping for the interview you're going to get is required. Don't expect "what is the difference between finally, finalize and final?"

Even Google non-programming questions still fall into the same "puzzler" style, just more in Google's style. Think "how would you work out which are the only two people in a room of people who can speak Spanish, when nobody has the same language as you (and you don't know Spanish)" stuff.

It can be used to gauge abstract thought, stress handling, interview preparedness - it is still a "good" interview question for what it says about you, the candidate, in terms of non-technical knowledge.

||improve this answer
  • Hi @Davor, your comments here do not meet standards of professionalism expected for the Workplace. Please read this link for more information if you have any questions. – enderland Nov 24 '14 at 14:37
  • 1
    @enderland - Oh I do have a question. Who determines what arbitrary standard of nice do we follow? – Davor Nov 24 '14 at 15:35
  • 1
    @Davor use this post if you have questions, comments, or want to discuss the policies for Stack Exchange and The Workplace. Additionally, if you have questions regarding "what should comments be used for?" please see this post or this more lengthy post. Thanks. – enderland Nov 24 '14 at 15:55
2

When chatting with one Googler about a year ago (might be more already) on their hiring policy he was sharing how Google at some point ran a crazy huge survey with lots of different questions through all of Google (or at least those who had been in their ranks for awhile). Next they had people fill out the same survey and they started systematically figuring out using raw statistics which questions most distinctly defined a Googler a Googler. Why? Because they figured that their current employees fit best in their company and thus questions that where answered by all the same where worthless to ask.

Now, assuming that this statement was a consequence of that research then we can conclude that at least brainteasers are worthless to differentiate between high level employees. I assume that it does allow differentiation between just any and a truly bad candidate, but the same could be observed in a million easier ways as well.

||improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.