Is it reasonable to ask programmer candidates to solve a task before the interview? [duplicate]

Question

When interviewing for a programmer's job, we have a simple programming task that HR hands out to candidates after the first (HR) phone screen. It's a a total of ~130 lines of code spread over 4 java files, mostly stubs, and we ask people to implement their solution in the context of these classes.

HR said candidates drop off after HR sends them the programming task. Some candidates complained the instructions were outdated (may have referenced a university class a few years ago). I can see that, but this wasn't the only reference. Others complained the instructions were unclear.

This is frustrating, cause the task is a simple implementation of a standard algorithm. There is an RFC and even a Wikipedia page that describe it. It seems to me that people who complain about references and instructions did not bother to do a web search. There is plenty of open source code that does what's asked, so people could research and even compare their code against others' code.

Is it unreasonable to ask candidates to turn in a small programming task within a few days or even 2 weeks? I'd think this task would take me about 4 hours and I'd end up with ~500 lines of code total. Someone not familiar with the task may need 8 or 12 hours, depending on how much they want to polish their code.

If it is unreasonable, how else would you gauge the candidate's programming skills? During an in-person interview on paper, whiteboard, or computer?

Answer 1

I'd be concerned with a task that required 8-12 hours of effort after just an HR screening interview.

There should be some relationship between the time that you've invested in a candidate and the time you're asking a candidate to invest in you. If I've just had a phone screen with HR and not even had a technical conversation and was asked to spend 12 hours on a code exercise, that would feel rather unbalanced. You've invested maybe 30 minutes of HR time in the candidate, you're asking for 24 times more time from the candidate. If I were a candidate, I'd expect that there were probably dozens if not hundreds of other people in the running for the position since basically no one gets excluded from an HR screen. Investing a couple dozen hours coding when I'm one of 100 candidates hoping to get a technical phone screen would be a relatively low value proposition-- I would expect that there would be a good chance that I'd invest a lot of time and never hear from the company. If the request came later in the process when there were fewer people in the running and the time investment was more equal, my calculus would likely change.

Additionally, if you are recruiting candidates that are already employed, they're going to be doing this after putting in a full day of work. They're probably not going to have long stretches of uninterrupted time, they're going to be doing an hour or two of work every evening. If it would take them 8-12 hours during the day when they're at their best, it might well take twice that when they're trying to piece together time in the evening after a full day between their other commitments. A candidate could do this for one company but if they're applying for a number of different jobs, it would quickly become untenable.

Of course, if you are a really well-known company that developers are clamoring to work for, you can put much bigger hurdles in your application process. SpaceX, for example, probably doesn't have to worry about scaring candidates away. There are enough really good developers that really want to work for SpaceX that they'll have more than enough to choose from. If you are a small widget manufacturer that no one dreams of working for, driving away candidates early in the process is likely a much bigger deal.

Answer 2

Someone not familiar with the task may need 8 or 12 hours, depending on how much they want to polish their code.

That's absolutely unreasonable. You're not paying them anything and yet you expect them to work for an entire day for free doing useless busywork just so you can feel comfortable that you know what they're doing.

Here's what you do (and what I've done). You make a test that I like to call the BS Detector. Ask questions that they should be able to answer. Vary the difficulty from simple to difficult. Don't sweat them looking stuff up online, that's fine. Have HR give the test out and send it back in a day or so.

Then you invite them in for an interview and ask them about the answers. Why would they use this method over the other one? When would be a bad time? Again, you're only checking for BS. Talk to them about projects they've done and the like.

There's no perfect way to discover if someone can code. You need to get them to talk about code and about what they've done and why. It's not objective. Make sure they're not full of crap. Trust your gut. Ask a lot of "how would you solve ...." questions. And have their experience verified.

Then you need to bring them in and give them a go. You'll know very quickly if they've got it or they don't. And don't be afraid to let them know that.

The simple fact is that you will never be able to completely avoid idiots. They'll slip through. You can them quickly and move on. Aptitude and ability is more important than whether they can do your algorithm.

Bottom line: I've turned down interviews because they ask me to do too much as an interviewee. I don't mind a few hours here and there but if you want to sit me down in front of a computer and write for more than an hour or so for free, forget it. I've left interviews before for that. It tells me that the company is treating me like they're doing me a favor rather than coming together for a mutually beneficial relationship of peers.

Answer 3

I don't know about other people, but if a company requests that I perform 4 hours of unpaid busywork for them before they deign themselves worthy to talk to me face-to-face, they go to the bottom of the pile of potential new employers.

The whole idea of "do this unpaid piece of busywork to prove that you aren't lying about everything on your resumé" doesn't seem to come up anywhere except with software jobs. I have no idea what makes people think that's it a good idea to ask potential hires to do this.

Next time you're looking for a new cleaner, request that they do your dirty dishes to prove themselves before you let them interview, see how that goes.

Answer 4

Is it unreasonable to ask candidates to turn in a small programming task within a few days or even 2 weeks?

If you search through The Workplace, you'll see many questions that come at this issue from the other side (that of an applicant). Many of them complain that the potential employer is getting hours of "free labor". And many complain that they are expected to invest too many hours before they are even sure it's a job they would want. That could explain why you see so many drop offs.

For me, it's always better to use a whiteboard than hand out a programming task to do at home. Hearing candidates think things through out loud has been the best indicator of competence in my experience. But obviously many employers use the same approach that you do.

I think the best way to judge the "reasonableness" of any interviewing approach is to judge its effectiveness. You find your current approach "frustrating". In spite of that if you are able to use your approach to weed out unqualified candidates and efficiently find good ones, then it may be worth the frustration.

On the other hand, if you feel it's not worth the frustration and you aren't finding enough good candidates, then you may wish to try different approaches and see if some work out better.

Answer 5

I don't think it's all that unreasonable except that you'll want to watch out for two things:

1. There are companies out there who issue these "tests" that actually require the programmer to do some unpaid work for them. It doesn't look like you're doing this in the slightest but please be aware that this exists and take steps to make it clear that it's not remotely what you're doing (by clearing out any reference you might have to real-world projects, for example).

2. This may not actually give you a lot of insight as to how well someone works but instead lets you know how well they can look things up on Google. I mean, don't get me wrong, that's not a terrible attribute for a potential employee to have (Google has saved me many times) but if you're looking for a person who can actually code and not look up answers then the test may not be all that informative.

To be honest I just think that a lot of potential programmers out there just aren't qualified for... anything. I'm not sure what leads a person who doesn't really have the basic qualifications to do a job to apply for it, and I have never really conducted interviews outside of the industry so maybe this isn't just a coder thing, but it does seem like the software dev track is kind of prone to people who apply for jobs and can talk a good game (sometimes) but don't actually have a fundamental understanding of how to write code.

I wish I had an alternative solution but I think the only one is really to use several different ones. An in-person coding quiz (doing something super-simple, perhaps, or slightly more complicated with easy access to the Internet) coupled with a good technical interview might do better than a take-home quiz alone. I will say that so long as the test isn't that hard, you're probably not losing a lot of people who might otherwise work out for you by requiring that up-front test.