4

I am a new interviewer and am trying to learn how to screen a good candidate from a bad one. I had a case in which the candidate looked up the answer to my question on the internet and after just changing a few variable names here and there, claimed that she suddenly came up with the right answer. The answer matched the published answer right down to curly braces for compound statements and tab indentation.

My question here is not how to prevent someone from cheating, rather if there is any use in disqualifying such candidates. It says something about their integrity (perhaps I have been a TA too long :) ) but as my manager said, they would be tested in the loop later anyway when they won't have access to the computer.

EDIT:

Thanks everyone for your response. Clarifying a few things since they were asked:

  1. It was a Skype based screening interview with the screen shared.
  2. I think it was pretty clear that I didn't want to Google the answer since I explicitly said I wanted to test how you think about the problem and can code it (but not Google skills). Maybe I should be more explicit about that, good point.
  3. Don't ask questions from the internet - this is an arms race IMHO. Eventually what you ask will land up on the internet, especially if the question seems good.
  4. As suggested, I did go down the lengths of questioning about concepts rather than just the code (before and after the code was copied). The answers came back the same as the comments written in the reference code :-)
  5. I think posting the question would detract from the topic, but the question was not a standard FizzBuzz-style question. When I gave the question to fellow devs, they solved it well within 20 minutes (or less).
  • 8
    Did you tell them not to look up the answer? – Patricia Shanahan Jan 5 '16 at 3:18
  • 51
    If you don't want answers from the Internet the don't ask questions from the Internet. – paparazzo Jan 5 '16 at 3:36
  • 31
    If they can google the answer then it's not a good interview question. – TheMathemagician Jan 5 '16 at 4:23
  • 4
    I'm happy enough for my people to use google when they need an answer fast. Just saying.... rather that than struggle and waste time for a day. – Kilisi Jan 5 '16 at 9:50
  • 3
    Did he/she admit the answer was from the Internet? What was the question? Was it a trivial question with a trivial solution (eg, write a bubble sort), or a common interview question (write a Fibonacci sequence generator), or something else very common? The smaller the scope of the question, the more likely it will be to get similar answers. There are only a handful of common curly braces and indentation styles in use these days. Opening brace on same line or next? Indent 2 or 4? Spaces or tabs? – Kent A. Jan 5 '16 at 12:37
35

An answer is only as good as the question that was asked.

This is unfortunately a case of asking a bad question that can simply be Googled like those famous "how many potholes" questions you see everywhere.

If this person was already hired and you asked them the same question, and they Googled the answer and provided it to you, wouldn't you be happy with that? This is how a practical workplace operates and the reason why sites like StackOverflow are so popular.

You do not want your employees to be solving problems that have already been solved or are trivial in nature. If a solution exists - especially if it has been proven, it is almost always better to use that solution instead of designing your own. Therefore, you want employees that excel at understanding a problem, breaking it down to its component parts, identifying the common solutions and finally spending time on what is unique with that problem for that specific domain or application.

If you hire a software developer to write an API framework - you do not expect him or her to write a HTTP library from scratch. But when interviewing developers for these positions, that is a typical question most interviewers will ask.


As someone who has also had to struggle to come up with practical and insightful interview questions for programmers - it is difficult to come up with something that isn't trivial (so that it can't be looked up) while still generic enough that you can observe the quality that you are trying to see in the candidate.

Here is an example:

  1. Bad: "Write a bubble sort".
  2. Better: "What is the difference between a bubble sort and an insertion sort?"
  3. Even Better: "When is the bubble sort not efficient?"

You want to get an idea for the depth of a candidate's knowledge and their thinking analytical process - not judge how good their "Google-fu" is. But many times that's what ends up happening with bad questions.

You should definitely not disqualify someone who is smart enough to know how to research a question - you should instead concentrate on asking the right questions for that particular candidate.

  • 2
    This is a great answer. There are too many people interviewing developers that are simply not asking the right questions. Personally, I don't want someone wasting their time writing code that can be found in a couple minutes. otoh, understanding the concepts is far harder and that is the knowledge I pay for. I did development for nearly 20 years and if you ask me to write a simple for loop I'll probably leave something out - that's what autocomplete and syntax checkers are for. But ask me to explain how to architect a system and I'll educate you. – NotMe Jan 5 '16 at 15:52
  • A nice question: When is bubblesort the most efficient method? I know one example. – gnasher729 Dec 2 '17 at 17:26
  • 1
    What's wrong with asking all three questions? You can ask the second two, but some people will answer those perfectly and can't write a bubble sort! – Chan-Ho Suh Aug 29 '18 at 2:45
  • 1
    I learned various sorting algorithms when studying Comp Sci. I haven't used a single one in *cough* decades as a professional developer. Thankfully, no one has ever asked me at interview :-) – Mawg Dec 11 '18 at 12:12
7

It depends the question.

If this was a FizzBuzz-style filter question, absolutely. Too many people, too little time. As a slight aside, if you're giving a question like this (recommended), it is probable that given a large enough group of people some of them will produce the exact answer with the exact syntax as the first Googled result. Especially if that first answer found is in 1TB or another common coding style. I've seen this.

If this was "know the obscure language feature or class", nah. If we're honest, a surprising amount of programming (i.e. coding, not design or architecture) is done by looking up the answer. And that is a pretty good skill. A large number of, supposed, programmers will make a hack-together-buggy-mess of hundreds of lines of code when the first answer on Stackoverflow is correct and five lines. But if they lie about possessing this skill, I wouldn't trust them on the skills they claim to have. Lying would be an instant disqualification in my books.

If this was a difficult problem, they struggled for awhile, then they found the answer online verbatim. Disqualified. If this was a difficult problem, they struggled for awhile, then they found the answer online and needed to change critical pieces (or they only liberated one piece of code into their solution), it is ok but not optimal. I deduct 'points'. That would be the most gray case.

  • 1
    Regarding FizzBuzz: surely you're disqualifying dozens of excellent developers who go "Oh hell, not another one of these boilerplate time-wasting questions.", simply look up the first answer, verify that it fits and submit it? And if candidates follow standard syntax/pseudocode coding conventions, FizzBuzz answers will always be close to identical, precisely because they're so simple. – Lilienthal Jan 5 '16 at 16:11
  • The point of fizz buzz is to is to test minimal comprehension. But if you give this test, please customize it. E.g., at minimum you should not mention "fizz" or "buzz" anywhere in your problem description. Including modulo and loops is a good idea, because this is minimal programmer-comprehension stuff. – Brandin Jan 6 '16 at 11:31
  • @Brandin I've heard it suggested to not customize FizzBuzz-style questions. Reason being that the stock question can be found with a good description whereas our many tweaks can make the problems hard to understand. There are rarer FizzBuzz-style questions (ex. find the missing number from 0-N in an unsorted list of unique numbers of size N), I'd suggest using those instead of changing FizzBuzz. – Lan Jan 8 '16 at 13:14
  • 1
    @Lan By "customizing" fizzbuzz I don't mean "tweaks" but to change the wording. For example, if your company is called "FooBar ltd", then you might replace "fizz" with "Foo" and "buzz" with "Bar". This is enough customization to test minimal comprehension. The computation part of the problem is precisely the same, but that change alone is enough to make the problem solution not trivially searchable. – Brandin Jan 14 '16 at 9:17
5

My question here is not how to prevent someone from cheating, rather if there is any use in disqualifying such candidates.

Of course there is a use in disqualifying candidates who cheat - you are removing the cheaters from your further interviews.

You characterize this as "cheating". And you are basically saying that this candidate lied to you ("claimed that she suddenly came up with the right answer").

So you have to decide - do you want to hire cheats and liars?

I know I don't. If someone lies to me, I assume that they will continue to lie to me. That's not something I'd want to see on my team.

  • 1
    This assumes that the candicate knew that looking up the answer is not acceptable; OP did not make that quite clear. However, if they were clearly informed about that rule and decided to break it, it's probably not a good idea to hire them. – sleske Jan 5 '16 at 15:11
  • It's not like the interviewee was hiding the fact that they had used Google -- they did the lookup on a shared screen! Had this person been intending to cheat or be dishonest, they'd have used a different device than the one they were screen-sharing with the interviewer. – LindaJeanne Jan 6 '16 at 7:17
  • @LindaJeanne They didn't look up the answer on the shared screen. Possibly a mobile or a different computer. – proteus Apr 1 '16 at 2:41
2

There is a level of integrity to be considered. I would follow-up with asking the person if they looked it up. You may find that they not only admit it, but they feel this is better than not getting the job done or taking an excessive amount of time and energy doing something that is routine (e.g. Fizzbuzz)

The real risk of the "copy and paste coders" is that they are applying a chunk of code inappropriately and possibly may not know how to adapt, adjust, debug, enhance. A brief coding session with some questions about there work should give you an indication if they still know what they're doing.

I write about 2-3 connections strings a year if I'm lucky-the next one is going to get copied and pasted with some minimal changes to get it to work.

You need to beef-up your interview and make it closer to the required task. Chances are, some internet searching is going to be required, so make sure candidates are doing it correctly. Few people can get by in a job just by looking up answers to trivia questions.

1

The problem you're going to face is proving they did "cheat".

If the problem is trivial enough to google, it's probably trivial enough to come up independently with similar solution to what's already been done by other people.

What you can do is get them to explain how the solution works - but won't definitively prove they "cheated" (or not).

At best, you can make a note that this person's solution is remarkably similar to the one on google, and talk to your manager about it (which you have done).

In the long-term - you should design tests that can't be googled.

  • 1
    I disagree with coming up with problems that "can't be googled." In this generally context. This was a phone interview. They tend to be "filter a lot of people, get rid of obvious rejects." Googleable questions tend to be easy to describe, easy to evaluate for filtering proposes, and easy to find. – Lan Jan 5 '16 at 3:06
  • 1
    Are you sure this was a phone interview? OP didn't say that. – HorusKol Jan 5 '16 at 3:10
  • "Phone Screen" I took as meaning "Phone Interview/Filter". But jargon does vary across the world. – Lan Jan 5 '16 at 3:11
  • 1
    They simply said "screen" - which could mean a phone screen, or some kind of pre-interview test. I wouldn't be looking for code examples with "variable names" in a phone conversation – HorusKol Jan 5 '16 at 3:14
  • You're right that I probably over projected the situation; filling in the gaps with my personal experience. I've used and seen online phone screening software, where the idea is a 15-minute phone interview while the interviewee logs onto some service to do a few coding problems in real time. – Lan Jan 5 '16 at 3:23
1

My question here is not how to prevent someone from cheating, rather if there is any use in disqualifying such candidates. It says something about their integrity (perhaps I have been a TA too long :) ) but as my manager said, they would be tested in the loop later anyway when they won't have access to the computer.

Has it been made clear before that "cheating" (aka finding solutions online) is explicitly not allowed for this interview question? I know interviewers that tacitly encourage this kind of thing, as a test of wits kind of situation.

If it was explicitly forbidden, then sure, you should disqualify that person. Violating interviewing rules set beforehand is a major red flag and signals that this individual is unwilling to even follow direct orders.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.