6

We are currently filtering some applicants for a technical position. We invited 4 applicants to take a coding test from a coding test site. 3 out of the 4 chose to copy solutions that they found (on stackoverflow) and submit them as their solutions. One didn't modify the answer at all, one modified it slightly, and the third made modifications to the structure of the solution but still based it on the on-line solution.

I've been looking around to see what the general consensus is for something like this, including this thread.

My personal, professional opinion is that this is grounds to disqualify the applicant immediately - however I'm not the only person evaluating this situation. Others are of the opinion that perhaps the test was too difficult for the position we were hiring. Another suggestion is that this sort of behavior may be considered 'acceptable' since a lot of us use resources like Google and StackOverflow to do our daily jobs.

Our 4th candidate did NOT copy and paste a solution from the internet, (s)he did original work.

How have others dealt with this? Would you give someone who did this as part of the job interview a second chance?

|improve this question
  • 9
    Did the instructions say to not copy solutions? – Chan-Ho Suh Mar 3 '16 at 0:56
  • 25
    If I can solve a technical problem by copy/pasting StackOverflow, I am going to copy/paste StackOverflow. "perhaps the test was too difficult for the position we were hiring." no - if you can find the solutions on StackOverflow perhaps the questions were too easy. – emory Mar 3 '16 at 1:20
  • 11
    If you don' t make the rules clear, that's your error, not theirs. Having said that, I would give a demerit to someone who copied an existing solution without (a) crediting it, (b) noting what license it was under, and (c) explaining the licence's rules of usage. Often what can be used professionally is not the same as what can be used for personal work, and what can be incorporated into a product is yet another question. Part of being a pro is understanding appropriate use of intellectual property and its impact upon a business. – keshlam Mar 3 '16 at 3:15
  • 21
    Also, if you don't want off-the-shelf answers , don't ask off-the-shelf questions. – keshlam Mar 3 '16 at 4:34
  • 7
    What if the applicant is the original author of the Stackoverflow answer in question? – Brandin Mar 3 '16 at 7:23
10

It appears that you've already made up your mind that what they have done is wrong, based on some of your responses.

Rather than making them take a different test, you could ask them why they answered the question the way they did.

If you were to do this follow up over the phone, it would give you a pretty clear indication of whether they know what the code does or not. You could ask them why they did not use X method, but then you run the risk of them only really knowing the one method, depending on the level of the technical position you are interviewing for. Regardless, it will give you a better insight into their abilities than you'll get from what seems to be an easy test.

You don't want them re-inventing the wheel, but you do want to know that They know how the wheel works. And as the above answer states, you didn't explicitly say for the coding to be original.

|improve this answer
  • Yes, I have very strong opinions on this, but my coworkers and I disagree, so I'm looking for a view from 'the other side'. – ben_979 Mar 3 '16 at 16:30
  • 1
    +1 for quizzing them on the code they've written. Talking with them and determining how much they understand the code they submitted is much more valuable than the code itself. – David K Mar 3 '16 at 18:07
  • 1
    +1 for having them explain it. That will separate the resourceful from the script kiddies. – Richard Says Reinstate Monica Mar 3 '16 at 21:46
  • 2
    We decided to begin re-interviewing some of the candidates. Part of the second interview includes a requirement to explain the coding question they were asked (to insure they understood it and didn't just Google the answer) and we are also giving them every opportunity to mention that they researched the answer. Also we are also applying a very simple programming test in-house on a problem we developed ourselves and isn't on StackOverflow (at least not YET :-) ). I appreciate all the feedback from everyone. – ben_979 Mar 7 '16 at 17:01
19

My personal, professional opinion is that this is grounds to disqualify the applicant immediately - however I'm not the only person evaluating this situation. Others are of the opinion that perhaps the test was too difficult for the position we were hiring. Another suggestion is that this sort of behavior may be considered 'acceptable' since a lot of us use resources like Google and StackOverflow to do our daily jobs.

I think it is a huge mistake to give job applicants a task and not make it clear what you expect out of them.

You weren't even clear within your own company what your expectations were - you cannot expect an applicant to read everyone's mind, take a mental poll, and come to the proper conclusion.

Rethink your interview process.

If you decide a coding test is still appropriate, evaluate the process and the test itself. Remember that you are only trying to distinguish good candidates from poor candidates - not to find out who is the best test taker.

Determine if the test level is appropriate for the job being filled. Determine if you want candidates to work from memory, or if they are permitted to use the internet, a paper cheat sheet, an open book, a calculator, a phone, or nothing. Then make it clear to candidates what is being expected of them, and enforce the restrictions. A coding test site may not be able to offer much help there - you may have to decide if you care enough to spend the extra money and have the tests properly proctored.

Periodically, review your process. Get feedback from applicants as to how they viewed the process. Make sure what you are doing still meets your expectations. And make sure that you are actually ending up with better employees this way - that should be the whole point of the exercise.

|improve this answer
  • That's the correct answer. As an aside, the OP should consider this batch of candidates FUBAR and start anew after re-thinking the process. – Deer Hunter Mar 4 '16 at 2:09
  • 1
    I completely agree. Part of being a good developer (or any employee, really) is to be able to seek answers to questions you can't solve alone. It's possible that your interivewees did not know how to solve the problem, which is probably a red flag unless it's something completely unknown to them (asking a SQL dev to write some low level C code) but it may also show that the person is capable of research and identifying valid solutions. If it's a short fizzbuzz question, perhaps it's not such a good thing. I would suggest asking candidates to explain why the code works in addition to coding. – Brian R Mar 4 '16 at 17:11
5

Using the internet as a resource would be fine with me. I would prefer the candidate who had their own solution, but realistically just because they didn't copy from stack exchange doesn't mean it's original either. It's not the only resource available.

However I ask them to cite sources if they do such a thing, and would judge the one that was modified to suit the question properly to be better than the verbatim answers.

This should not be the only criteria to judge a candidate on. I would look at all aspects of their competency and fit for the company.

Realistically when they're working for me, I want them to solve problems as quickly, robustly and efficiently as possible, not reinvent the wheel every time if it's going to cost me. I don't really care how they do it. So I give them technical questions more to judge their knowledge than to solve a problem they can easily look up.

|improve this answer
  • I agree with a lot of what you are saying, except as I said in an different comment, I think that copying and pasting shows me nothing about their capabilities as a software developer - and they should know that's what is being evaluated. – ben_979 Mar 3 '16 at 1:54
  • yep, that's why I wouldn't pull a question straight off stackexchange for them to solve. You have no knowledge of how they got their answer otherwise so it doesn't really 'prove' anything. – Kilisi Mar 3 '16 at 2:21
  • 6
    I would have given big bonus points to someone who acknowledged the cut-and-paste. But real programmers cut-and-paste code all the time. We used to do it from samples or manuals, but now stackoverflow is the missing manual for almost everything. – jimm101 Mar 3 '16 at 3:08
  • 1
    @jimm101: I agree. The number of comments with links to SO questions in our code base is almost funny. – NotMe Mar 3 '16 at 15:58
  • 3
    @ben_979 Being able to find and identify the correct solution is actually a large percentage of what it takes to be a good software developer. The solution that's marked as correct is often not correct at all or not as correct as another solution that the OP doesn't have the experience to know is "more" correct. – Amy Blankenship Mar 3 '16 at 17:17
5

This really depends on the requirements of the coding questions. Did it say explicitly outside sources would NOT be allowed and the solutions must be their own work? On the job, a lot of people will use/copy/modify code from StackOverflow, but, for job interview, this is not very ethical, because it does not showcase your actual programming abilities.

I would take a look a the solution from the candidate that did not copy from the Internet. If that solution is still not satisfactory, have all candidates take another coding test with the restrictions clearly stated.

|improve this answer
  • It did not explicitly say no outside sources allowed - but I don't think it should have to. As you stated, I don't find this behavior to be very ethical. A re-test is one of the options being considered. – ben_979 Mar 3 '16 at 1:56
  • 2
    @ben_979 "no outside sources allowed" -- so if an applicant consulted a reference book on a language specification or algorithms that would be a disqualification? – Chan-Ho Suh Mar 3 '16 at 15:32
  • 1
    Would "no outside sources allowed" include man pages? – jamesqf Mar 7 '16 at 6:05
  • 1
    That depends on the scope of the test. Generally, man pages and other official references should be allowed. I recommend enumerating allowed resources for the candidate. I have been quizzed on things I could have easily looked up, but the interviewer thought it was important to have it memorized. – jcmack Mar 7 '16 at 22:13
5

The tests are a medium to test the coding skills of the participants . Common sense says do not copy paste. The participants went ahead and copy pasted the solutions. This indicates that they don't have the common sense of avoiding copy paste solution. The one who did not copy paste demonstrated skills plus common sense. It does not matter if the test was easy or difficult. The approach could have been reviewed even if they failed the test.

My opinion , the three go out the door. The 4th one makes it to the next round.

|improve this answer
5

The third person who used stack overflow seems the best to me. Finding that a solution has already been solved but modifying it to match your scenario is surely the ideal developer you want?

If you want to test people's ability to code something entirely by themselves, get them to do the test without internet access (maybe just with access to the reference for the language, although A decent IDE would help with that)

|improve this answer
3

3 of your candidates demonstrated an ability to locate a website (StackOverflow).

1 demonstrated an ability to actually write code.

What more do you really need to know? Whether it's "wrong" or whatever else is completely beside the point - they were all given an opportunity to demonstrate their coding abilities, and three of them failed for that particular case.

I would argue that there's nothing wrong with using internet resources for an interview or application, as we all use web resources for our actual jobs. In fact, I personally favor giving candidates a laptop and a set of tasks to perform as a final technical interview (for systems jobs) - however they get it done doesn't much matter to me; the point is for them to demonstrate that they can do what is typically required for them to do the job. That same argument could be made in this case (they all came up with code that would work), but it seems to me that only one of them demonstrated what you were looking for (the ability to write code), so there's your candidate that moves on to the next round.

On a related note, perhaps your coding test needs to be reworked, if the answer is freely available on the world's most trafficked developer website. It doesn't sound like you're getting much value out of it. At present, it sounds like you could probably probably save everyone a lot of time by replacing the coding test by asking candidates if they know about StackOverflow.

|improve this answer
  • Why would I spend time writing code when I can do the job in less time by finding & modifying existing code, whether my own or someone else's? How is finding & using existing code to solve a problem different in principle from calling a library routine? As for giving candidates a laptop for a test, how many people can really work effectively without a standard keyboard and decent size/resolution monitor, not to mention their perferred toolset? – jamesqf Mar 6 '16 at 4:48
  • 2
    @jamesqf A valid argument once you're employed, and why I don't think it's "wrong" to copy code for a code test interview. Probably not smart, though, as this question demonstrates. The purpose of that question, after all, is to test coding ability, not the ability to search SO and copy from it. As to the laptop test, it's mostly for weeding out the bullshitters, so nothing especially difficult. (And I give one chocked full of tools, often mine.) "Oh, so you've allegedly been working with vCenter/vSphere/ESXi for 8 years, but you can't consolidate a snapshot? Suuuuure you are." – HopelessN00b Mar 6 '16 at 6:01
  • You don't have e.g. several decades (or even 8 years) of custom editor macros? And I'm going to be dead slow without a standard cursor keypad. As for whether it's good to copy for a code test... Well, that depends on what the test is supposed to be testing, and as this question & responses show, sometimes you just have to guess. I can imagine a different poster asking what to do about the one candidate who re-invented the wheel :-) – jamesqf Mar 6 '16 at 19:18
  • @jamesqf I kinda the same way, so I have a Logitech keyboard/mouse that accompany my laptop anywhere... but like I said, it's more of a way to weed out the bullshitters, so there's very little in the way of things that would require a macro, and "dead slow" is just fine. I've been in heavy Windows shops for a while, so there's a lot of looking up whatever cmdlet or option or API on Technet, which is both acceptable and expected. – HopelessN00b Mar 6 '16 at 23:49
  • Well, sure, but the way I do that (Linux, so it's man pages) when I'm writing code is to hit Alt-m on a function, which opens the correct man page (or in some circumstances other docs) in the editor, so I can copy & paste from it... – jamesqf Mar 7 '16 at 6:10

Not the answer you're looking for? Browse other questions tagged or ask your own question.