Should I use a technical challenge when hiring for a junior position?

Question

My team is looking to expand and we are ready to hire two junior developers to become permanent members of our group.

We won't mind a graduate just off college with a basic understanding of databases, HTTP and basic web programming as in the end we're looking for fast learners and commitment rather than experience.

My dilemma is: should I request a candidate to do a technical challenge such as a programming exercise? I don't mean whiteboard coding, but rather some homework to show how they would approach a problem without the stress of people looking over your shoulder.

However, I think such an exercise is useful mostly to assess experience rather than ability to learn. Furthermore, I suspect many junior developers would be terrified at the mere idea of doing a programming exercise for an interview (ever heard of impostor syndrome?), and I'm afraid we'll miss some talent. At the same time we obviously can't afford to hire someone with no experience as a programmer whatsoever, and I'd rather not do whiteboard coding.

At the moment I'm considering a mixed solution, i.e. to either require contribution to open source projects, or a GitHub repository, or a Stack Overflow profile; or do a technical challenge.

Should I use a technical challenge when hiring for a junior position?

Answer 1

If you want to have coding demonstration, without a homework exercise or whiteboard, do a paired programming exercise.

This can be a couple different ways.

1. Debug/troubleshoot examples containing obvious, non-syntax bugs. Write up a somewhat small example (something someone can look at and work through in a manner of minutes, not hours), with very deliberate bugs/mistakes of varying difficulties. Have the candidate talk through the code, explaining what issues they find and how they'd fix them. Keep these conceptual, not syntax (not "missed a semicolon here!" but more "loop incorrectly setup" or "not cleaning up memory" or "not threadsafe" issues). These issues should be core programming issues of varying difficulty to identify.
2. Write a basic application. Have the candidate write a basic application to do something. You can make it relevant to your business, or not.

Make sure to tell the applicants this is going to happen. This is going to very much fluster/intimidate introverts and throwing it at them without advance notice is not a good idea. You might even tell them the rough concept of the examples.

During this, make sure you:

• Ask questions to guide and determine the skill level of the applicant. Don't let them go for an hour with no feedback.
• Recognize people will be nervous. You might even acknowledge that. "Hey, this might be uncomfortable, so let me know if there are things I can do to make you feel more at ease."
• Give at least an hour of time.
• Have a good setup (dual, mirrored, monitors)
• Ensure tools don't get in the way. Don't drop applicants in a really complicated IDE setup
• If the candidate is stuck, give feedback/help and don't let them sit there spinning their wheels. This wastes everyone's time.

Answer 2

I'm a junior engineer who has been working full-time for a year, and I've gone through two interviews (and got both jobs). I will share what the first company did that I thought was good and got us our good hires.

Firstly, in my opinion, don't give "homework". You are only going to get the people who are desperate for jobs to complete them and turn them in. Anytime a recruiter calls me now and gives me homework before I am even given a phone interview, they don't hear back from me. I'm not in school anymore, sorry.

Anyway, they brought me in and gave me a multi-page quiz on various technical topics related to Unix, Java, and object-oriented programming. There were also some very simple physics and math problems on there. After that, there was a simple whiteboard coding exercise (and I mean very simple, no classes or OOP or anything, just something with strings).

In my opinion, it was a great way to interview and the company had all very smart people working there. You would be surprised how many people who have college degrees in programming can't even write a simple for loop or understand strings. I think being able to think on your feet is a great ability for an engineer, and shows whether you crack under pressure or rise above. More importantly, it shows that you prepared for the interview. If the candidate prepared they should be slightly nervous but not enough that they forget how to program. The only caveat to this is it requires you to remember perfect syntax, which is kind of useless in today's world of IDEs. Use your own discretion for that.

If you're curious, the company I work for now had a much easier technical interview. They basically asked how I would go about designing a piece of software for a specific function, as well as simple questions on multithreading. The people are just as smart here if not smarter, and I enjoy the work way more. It seems like it's mostly a crapshoot in this industry.

Answer 3

should I request a candidate to do a technical challenge such as a programming exercise?

Absolutely.

Think about it this way: you are there to evaluate someone's ability to do the job. Their job is to write code. It's not to answer trivia. It's not to be a good cultural fit. It's not to go around telling people how they did stuff in a previous job. So why wouldn't you have them actually write code? Are you sure you can evaluate how well people can write code without it?

I suspect many junior developers would be terrified at the mere idea of doing a programming exercise for an interview

In my experience with senior level CS students and interviewing programmers for 10 years, this is very often because these junior developers are incompetent. They're scared to death of writing code because they know they'll be found out to be the skill-less code monkeys (or outright cheaters) they are. And that's good! As an interviewer, that is what you are supposed to be doing.

Sure, you'll get the occasional case of impostor syndrome, but that's something you the interviewer can help expose. And you should certainly try to reduce the stress of the interview process - including any coding problems. That does nobody any good.

In my experience, the best approach is to sit the candidate in a cube/at a desk with a laptop for a timeboxed on-site coding problem. Something like an hour to solve a few role-appropriate problems.

Why? This lets you give them requirements, and allows them to ask questions. They'll have to do that in the job, so make sure they can do it. It lets them see what your working environment is like. It helps prevent cheating (since you know they wrote the code and you can look at browser history to see what they needed to look for, and if they copy/pasted code). And we've found that having candidates code for an hour before interviews helps calm them down and get them in the right frame of mind for questions (often about the coding problems) later.

Oh, and by having a handful of consistent programming problems, you can more easily compare and contrast your candidates, apples to apples.

Answer 4

Yes you should test them, but on premise. Homework is useless. It shows that they have talented friends/family or were lucky and found code or whatever. It in no way shows their skill.

I have given similar entry level database tests. They are more word problems and relationship models then they are creating tables and doing some form of sql.

For instance I would say we have trainers that teach classes, some of these classes are in person, some of these are via web, some classes only some groups of students can be in, some classes anyone can be in, some trainers teach everything, while some trainers teach only some...

Please draw out all of the database tables with as many fields as you can include to get all of the information about trainers, students, and classes. Then please draw out how you would input these things into the front end.

There isn't a right answer because there isn't ALL the information. It is a very open question and you will see how far candidates take it. It is very easy to teach entry level people the syntax needed for your web pages, how to create tables, and add fields. Some people though have a hard time grasping data normalization, efficiency, relationships, and inclusiveness.

Answer 5

Make it Relevant to the job!

It does make sense to give a bit of a programming test, but do make it something that is relevant to the task at hand. I once did an interview where I was asked to implement Conway's Game of Life. It was pointless! Yes, I was able to do it, but I almost walked out, as it had nothing to do with the real job (this was interviewing at Williams-Sonoma).

Asking someone to do something relevant to the organization can open up all sorts of domain-specific insight they may have. I think that would help pinpoint the best candidates. They may also have some approach that you have not considered towards solving a long-existing problem.

Answer 6

I've seen this several times. When I was more junior, one company asked me to troubleshoot a page (in production, which seemed odd but I was naive) and offer suggestions I would implement if I were hired.

You guessed it - they turned me down, fixed the page based on my feedback, and implemented some of my suggestions. What I thought was an interview turned into free labor. This was several hours of my time, for which I wasn't compensated. Please keep this in mind before you assign "homework" to a candidate.

You may have better luck with some kind of online quiz. I know there are a bunch of these (I don't know if BrainBench is around anymore, but when I drop that name most people know what I'm talking about)

From the interviewee end, I'd say i possible send the quiz FIRST, and based on their scores bring in the people you want to talk to (if the quiz is important).

I'm a senior level developer now, and I'm still asked to take these after an interview (even when the interview is fairly technical and seems to go well). I always thought it'd make more sense for a phone screen, if that goes well send the quiz, and if the score was in line with what you need, bring in the candidate.

Answer 7

Be very careful about the type of "homework" you assign to a prospective employee - despite the fact that they've all been through college to get a degree in the same field, their backgrounds are going to be very divergent and different. A test for one skill will test for that specific skill and that skill only - and only for a specific case.

It is good that you're looking at an prospective employer's ability to study and seek out a solution on their own - since that is what most developers will do on the job anyway - but no matter what test you design, there will be bias, depending on how the test is written. Some people are terrible at the classic whiteboard challenge, while others would beg to have it back instead of what you're proposing. This is a risk you will run with any 'test of skill' you give to your employee.

The important thing, therefore, is to make sure that the skill you are testing for is a necessity for the job - if you want to test someone's SQL-querying, make it an SQL Querying test (note - this IS different than SQL manipulation), if you want to test their error handling, test their ability to interpret errors.

Just remember that the ability to pass one type of test over another does not determine the employee's general programming ability, or their willingness to learn. For that, you will still need to rely on the interview.

Answer 8

My answer will consist of two approaches. First two bullets, I will discuss how to improve the testing process to alleviate the downsides that you or others brought up. Then, I will devote a bullets to things you can do to improve your technical challenge efficacy.

1. It seems that you like the idea of an on-site technical challenge except for one caveat:

   Furthermore, I suspect many junior developers would be terrified at the mere idea of doing a programming exercise for an interview (ever heard of impostor syndrome?), and I'm afraid we'll miss some talent.

   This caveat is extremely easy to dispose of, however:

   Simply tell them honestly and upfront that what you're evaluating for - that is, you want to see how they think during coding process and what their style and approach is. That is, there's no pass/fail in the test in the sense of you have to produce a program with a given "correct" result.

   This will (from personal experience) calm down the

---

2. Some other answers mentioned a possible downside to a off-site "homework" technical challenge, which is that there is a risk of a candidate getting help that shows their work as better than their real capability.

   There are 2 easy ways to easily detect that in the in-person interview post-homework:

   • First, during the interview, ask them to slightly expand on their homework solution, by extending requirements. Someone who sweated their answers themselves would be immeasurably better at this than someone who were given the answer.

   • Supplement that with a smaller onsite coding test (whiteboard or better yet, pair programming in front of your IDE of choice).

3. A very good approach to a technical challenge is not to make them write code, but to review code. Give code some sample code that has inefficiencies and bugs of varying obviousness, and have them go over the code with you.

Answer 9

Several questions rolled into one:

Yes you should set a coding challenge, but no, do not set it as homework, for many reasons, some of which are to prevent cheating, copying, plagiarism, getting help, impersonation. But also because it robs you of two valuable parts: you get to see how their thought process works when under stress, and also (it's a two-way street) they get to see how you approach code development. For example, do you set a slightly ambiguous problem statement and expect them to make simplifying assumptions, or to down tools until you supply a definitive clarification? (the "right" approach depends entirely on your domain e.g. programming life-support systems is different to web code).

I think such an exercise is useful mostly to assess experience rather than ability to learn.

Even more fundamentally, it's to assess the thought process behind the code. Be clear whether you're assessing (algorithmic) knowledge or coding proficiency - those are two distinct things. Tell them you're not waiting to pounce on mistakes, just to see how they approach things.

Do keep it relevant to the job function, but don't ask for free consulting (as @Tim said - this does happen, with sleazy employers). Try to avoid toy problems like FizzBuzz, Towers of Hanoi, Conway's Game of Life. Pick something with more than one algorithmic component to make it immune to simple copy-and-paste.

However, Furthermore, I suspect many junior developers would be terrified at the mere idea of doing a programming exercise for an interview (ever heard of impostor syndrome?), and I'm afraid we'll miss some talent. At the same time we obviously can't afford to hire someone with no experience as a programmer whatsoever, and I'd rather not do whiteboard coding.

Your fears are misplaced. Encourage them to ask questions and/or document assumptions. Tell them it's ok and encouraged to use Google/ StackOverflow/ whatever.

Another good practice sometimes used (esp. on Craigslist or mailing-list/web-based job ads) is to email/post candidates a very simple problem, the solution to which they have to attach to an application. This is to weed out timewasters and people with no interest or experience or motivation; but again can be used to give them a flavor of what sort of code they'll have to write.

At the moment I'm considering a mixed solution, i.e. to either require contribution to open source projects, or a GitHub repository, or a Stack Overflow profile; or do a technical challenge.

For people who already have demonstrated proficiency by either of those, then either pick a harder problem, or describe them your problem domain and current issues and jointly define some problem with them - this also gives them insight into the job. (Or you could just ask them the standard question and let them breeze through it or whiteboard describe how to do it, but that's wasting both of your time and not challenging them - Joel and others advise don't.)

And we've found that having candidates code for an hour before interviews helps calm them down and get them in the right frame of mind for questions (often about the coding problems) later.

Yes. As long as you supply the guidelines above about how to approach it and what is being looked for. Not just treat it like some binary pass/fail filter with undefined criteria.

Answer 10

I think you should definitely set up programming questions in the interview. I'm currently doing a one year Placement and my manager told me when they were interviewing for the current position I'm at, they had applicants who didn't know what arrays were or how to use a for loop!

I would stay away from pre-interview exercises as they can easily find solutions online (i.e. asking your exercise questions on Stackoverflow).

IMO Coding Bat has a good set of questions for Junior developers (I was able to do these before I started working), so that you can filter out the completely incompetent ones from the competent.

Regarding databases I would say query them on simple SQL statements such as Select, Update, Insert etc.

Furthermore, please understand that the majority of these Junior Devs. have no experience in interviews and it will show (nervousness, making simple mistakes etc.). Don't let this be the main factor in your final decision.They should however, have the ability to show you that they're competent programmers regardless!

Answer 11

I'll risk giving an unpopular answer…

I see no point in testing at all.

Just hire them with a probatory period, which allows you to fire them for any reason or for no reason at all during the probation, and put them to work immediately as if they were members of the team.

The probatory period should be long enough to give you an idea about werther you like them or not, much better than any test. The only thing that a test will measure, is how much they are good at doing tests… but you don't hire them to complete tests, you hire them to actually do some real work!

If you problem is picking among multiple candidates, just use all the data you have: their CV, their interview, their published code.