12

We are having interviews for a new developer position. During the interview, I give the candidates, who tell us they already have experience in .NET and SQL, certain programming tasks, which I think should show the most basic of proficiencies if completed in about an hour.

Since I am the only developer right now, no one in the company is able to tell me whether the tasks are suitable for an interview or not.

During the tasks, I am available for the candidates' questions. I tell the candidates that since the questions are dependent on each other, they should ask if anything is unclear, and I help them if they are stuck, expecting them to be able to at least complete the next step. I also tell them that if they have problems with the syntax or with typing on an unfamiliar keyboard, they can instead discuss with me how they would implement it.

For some reason, all our candidates failed to complete more than half of the steps. How do I determine if the tasks are too hard, or if we just got unlucky with our candidates?

  • 5
    You're not going to get much value out of this, you're telling someone how to solve a problem you've given them, and asking them to fill in the blanks. It won't show you how that person thinks / solves problems / comes up with solutions – Joe Feb 24 '17 at 13:30
  • 7
    Do the candidates that failed ask questions? Do they sit at the screen and then just give up after some time? – Brandin Feb 24 '17 at 13:45
  • 5
    @Brandin nvoigt's answer was posted 20 minutes after I made the edit. Additionally, pre-existing answers do not change whether a question should be edited to be made on-topic or not. Our help center specifically states that you should avoid trying to answer questions that are not about the workplace. – David K Feb 24 '17 at 17:42
  • 4
    This would also be a good question for chat as it's more of a discussion than an actual question. – enderland Feb 24 '17 at 18:25
  • 18
    Having just come upon this question and having no interest in a discussion, I just want to point out that editing someone else's question to remove details that were there during most of the commenting/answering effectively breaks the process for new visitors. I have no idea what most of the answers or comments are about, and I am left feeling the question is too vague because it doesn't provide the details I'm looking for. – Darren Ringer Feb 24 '17 at 19:05
28

The coding part of the test took me 5 minutes, it would have taken maybe 10-15 if I had typed out all the pointless properties and assignments and implemented the method (that's never called). Double that for a potential crappy environment and interview nervousness and the time is fine. The SQL one should be just as easy.

I think you could make this significantly easier by omitting all the pointless properties the person needs to type out. If the person can implement and assign one property, don't make her type out the code for all of them.

Apparently, it works as a form of FizzBuzz. It weeds out people without a basic understanding of the language. That's a good thing. Just imagine you had hired one of them!

The fact that people fail your test shows that your process up to the interview is flawed.

You should ask yourself how it comes that you get to interview people that fail such an easy test. Did they lie on their CV? Is this common in your region? Is there a way to find out before? Maybe you can find a few simple questions you can ask on the phone? Or did you invite the wrong people? Did they actually say they knew C#, or did you assume that? I'm afraid this is where I cannot help you, because where I live, people generally do not lie or exaggerate in this way on their CVs. If they say they can code in C#, they can.

Please note that once you get to the stage where candidates pass this very basic test, you may want to refine it. Being able to do the basics is cool, but maybe not enough for your company.


As to the edit of the question how to find out if something is good as a test, just do what you did: ask your peers. Whenever there is a test to be prepared, I always grab members of other teams and try it with them. Have a few people do it and include the feedback they give on time and difficulty. As you are the only developer, you will need another way to find peers to do this: maybe a user group, your old classmates or a forum.

Do it as you would deliver software: Gather requirements (what do I want to test the candidates for), plan a test (prototype), let testers do it and gather their feedback, then refine the final product as many times as you see fit.

  • 3
    This is a good answer for the OP's problem but not likely to be useful for future users. Perhaps you could explain how the OP could validate your results himself to make it more widely applicable. – IDrinkandIKnowThings Feb 24 '17 at 22:21
13

I'm going to answer this question based on the information before the edit, as I personally feel it was still vital to the answer.

Have you heard of Pseudocode?

When I got hired for a software job I only managed to get half the tasks working. Mainly because I had to code in a language I never saw before and the syntax was different (I told them beforehand and they told that wouldn't be a problem). So for every piece of code that I couldn't get working I wrote some pseudocode.

example:

// Place a for loop here, For as long as i < [var].Length then do this other thing. 

The most important thing is that you get inside the software engineer's mind. Know how he intends to code. After that you need to ask yourself how easy it would be to bring him to a level where he can be placed to work on larger projects. Usually this will take between 1~2 months anyway.

Furthermore, I would HIGHLY recommend to let him use the internet for code referencing. Coding requires a HUGE amount of looking up codes in order to learn them.

Once he's done, and you're worried he just copied some code he doesn't understand, just ask him about the code itself. Why did he use this variable here? Why did he name it like that instead of fully spelling it out? Why use a for loop instead of a while? etc etc. These might be silly questions, but if he's bullshitting you, he shouldn't be able to answer any of this.

Also, try to make your steps as independent as possible and then leave him to his thing while you grab some coffee and do other stuff. If he gets stuck on anything, he can simply skip it and proceed with the next. A lot of coders don't perform well if they are being watched. So by enabling him to do his own thing, he should have that chance. I'd still pop in every 15~30 mins in case he has any problems/questions. Also, this allows you to see if he's a coder that will keep asking questions before trying to find answers online first.

As for the SQL tasks, I feel like you were too easy on them. I'd show them a small SQL database model and tell them you want to add a certain item, but don't wish to edit any of the current tables. Asking him what his suggestion would be in order to implement the change. And also why. basically talk the information out of him. I'd still ask how to make an update/insert/create query though. He can find that stuff within 2 seconds, but if he can't it's an easy way to see if he's bullshitting you.

Bottom line is, you need to understand how he thinks. Not how good he is at coding.

Also, you may want to tell him before the test, what kind of things he can expect (not in detail, just the outlines). That way he may prepare himself better.

  • 2
    This would be a great answer on programmers it is not a great answer on the workplace. – IDrinkandIKnowThings Feb 24 '17 at 22:22
  • 3
    "I'm going to awnser this question based on the information before the edit," Why would you do such a thing? That dues not make this a more useful question to future visits, in fact it does exactly the opposite. – user30031 Feb 25 '17 at 2:49
  • 1
    @Migz that is not considered important on Stack Exchange; the answer was changed because it was a bad fit. You are unfortunately disregarding a core site policy with this action. workplace.stackexchange.com/help/editing – user30031 Feb 26 '17 at 15:33
  • 1
    @DoritoStyle any question that changes the original question is not a good edit. In fact, the only thing the edit did was REMOVE information, not add anything. and then changed the core question. making it a completely different question in the process. I'm here to help the people who ask questions. Not to abide by the rules set by the community. In the end we're here to help people. Hense why I placed a disclaimer at the start of my awnser. I did it the correct way as it may have provided helpful information to what he was looking for. In the end, it's simply an opinion. – Migz Feb 27 '17 at 6:49
  • 1
    @Migz I absolutely disagree. Of course we're here to help, but we don't want to solve specific problems for one person, we want questions that will be useful to future visitors with similar general problems. If you can't abide by those guidelines then you are probably going to have a bad time on this site, since that is its absolute core purpose. workplace.stackexchange.com/help/how-to-answer "Not all questions can or should be answered here. Save yourself some frustration and avoid trying to answer questions which... ...are not about the workplace as defined in the help center." – user30031 Feb 27 '17 at 15:40
9

I think you have gotten a lot of great feedback related to the specific questions you asked the other candidates, but I don't see any real answers that address your stated question: How do I tell if a set of interview questions are too hard?

One obvious way you can answer this question is by soliciting feedback from other developers. You said that you are the only developer in your organization but that doesn't mean you don't know or can't meet other developers. You can ask for feedback from friends who don't work at your organization. If you don't know anyone else that slings code for a living then you could go to a meet up and try talking to someone there about the task. Finally you could consider running these questions past a recruiter. Of course there may be a conflict of interest there if that recruiter starts sending you applicants who perform amazingly well on your test but don't appear to know anything else...

Another direction you could go is search for interview questions online. This does two different things. First it might give you ideas about the style of questions other people are asking. Secondly you can compare the relative level of difficulty between common questions and the questions you're asking. You might think that this route might encourage cheaters since these questions are "known" but anecdotal evidence seems to show that even though FizzBuzz is incredibly well known a significant number of people still fail when asked.

Finally, I would argue that "too hard" is really relative to the kind of person you want to hire. Don't dumb down your questions because you feel bad that people aren't passing. Instead set your questions at a reasonable bar to screen out people who aren't going to be successful in the position. I'm certain that a position at Google where the applicant will be designing advanced search algorithms isn't going to have simple questions like "What is one difference between a POST and GET request?" Instead I'm sure they are asked advanced algorithm and machine learning questions. Having questions that are too hard for people who aren't going to be successful, but demonstrate an understanding of the principles they will need to be successful is ideal.

So my advice is to not worry too much if the questions are too hard. Instead worry about what are the basic principles you are trying to screen for that a candidate needs to be successful. Once you've isolated that the only real hurdle is making sure your questions are clear. That being said in my opinion there is real value to finding out if a person will ask for clarification before building a grand solution that solves the wrong problem.

2

What do you want this development task to achieve?

You have a small window in an interview, to judge whether someone has fed you a load of rubbish on their interview.

If someone was able to complete your task, would you then use that to think they are technically capable for the job?

The stuff you are asking seems to me fairly pointless to be in a technical test. You are getting them to create a class, a method, write a couple of lines of code and then do some basic SQL tasks. It isn't really showing knowledge in my opinion.

A better approach may be to show them code and ask how they would improve it.

For example:

I have a c# array of objects and I am doing a foreach loop.

You are then waiting for the candidate to say: "Oh, convert that to a list of objects and you can then use linq to search the list.

You can show them a SQL Statement with a clear inefficiency and then have an idea of an answer to be: "Oh, add an index on the table, or change that where clause to whatever.

You could even have an incorrect join in the SQL and see if they pick it up, or something like that.

They may also give you a completely different answer that does/doesn't work and opens up conversation to other areas where they can demonstrate knowledge.

Obviously tailor the questions to work around that example, but the above, I think, has the opportunity to provide further insight into how someone thinks and prove they know basic code etc.

  • 2
    But your example doesn't really make sense. Why would I need a MyDate when I can get all that info from a DateTime object. You are trying to work out how good they are as a developer. Rather than sit them at a PC and go through a "Colour by Numbers" exercise, get them talking about coding. In fairness though, your SQL task is much better. Maybe you just need to rewrite the task to a more real life scenario? – Andrew Berry Feb 24 '17 at 14:01
  • 3
    And immediately, by talking about code rather than a simple example, we are talking in a lot more detail about knowledge of .net rather than you watching me create a class in VS. Discussion about code/approach provides more proof of knowledge than tying a basic example. From that response I could say well you don't need a MyDate class to format the date time in a particular string format as we can do that with a string formatter. These sorts of discussions show understanding of the language. Completion of the task you had before, for me, didn't show that they could "program" – Andrew Berry Feb 24 '17 at 14:13
  • 3
    "Oh, convert that to a list of objects and you can then use linq to search the list" I'm not sure I would hire a candidate that wants to refactor code that's working fine, based on a sketchy understanding of another technique. LinQ works on IEnumerable, so it works on an array just fine. But you never said your foreach loop was read-only, surely a requirement for something named Language-Integrated Query. – nvoigt Feb 24 '17 at 14:14
  • 2
    @nviogt haha, my bad. But the underlying point is still there. If I answered with that, you could raise that as a red flag. That is more insightful than me typing out a couple of dozen lines of code which doesn't achieve much – Andrew Berry Feb 24 '17 at 14:16
  • 2
    I agree that parts of the test are repetitive and therefore pointless. But candidates should not fail it. As long as people fail it, it has a place. It's sad that it does, though. – nvoigt Feb 24 '17 at 14:20
0

I have two comments:

1) DateTime's SUCK to work with, if you want to see how people might handle converting from one data type to another or how they would deal with creating an implementing a class then I would suggest finding another example to work with. Additionally, what you're describing may be more a case of something infrequently done than difficult. In situations like that a decent developer should be able to resolve the problem within minutes using Google.

2) Creating and extending a table/schema are rudimentary and anyone that's ever written a SQL query should be able to create one to return records like you're asking, so if people are failing at this then either it's a syntax issue (used to having a reference) or they haven't worked with SQL at all.

One suggestion I'd make is that instead of giving them specifics to present them with the scenario and see how they choose to handle it. For example, give them the task to design table/s for a company that include employees, managers, and a way to query for the organizational structure.

Personally, I would find it more useful to know whether a candidate would jam all the info into one table or break it down (and why). Whether they can write a basic query or create a table isn't very useful information, how they approach design is much more important IMO.


Bonus: don't forget that some people REALLY struggle in an interview setting. I have known people who were very comfortable interviewing and answering behavioral questions but would completely freeze/blank on technical things because they weren't expecting it or prepared for something else.

IT is such a large field that you can spend weeks studying all the most common quiz questions and coding problems only to be given the one thing you overlooked... that does awful things to people in an already high pressure situation.

So keep in mind that part of your job as someone interviewing is to be able to figure out based on the way they approach questions whether it's nerves or a lack of knowledge causing someone to have a hard time.

  • 2
    Don't quiz - test... – HorusKol Feb 24 '17 at 14:23
0

How do I determine if the tasks are too hard, or are we were just unlucky with the candidates?

  1. How long is your interview? (For the sake of discussion, let's assume it's two hours long.)
  2. How many programming tasks are you intending to quiz the interviewee on? (Let's assume it's four tasks).
  3. Given four tasks for a two hour interview, could you complete each task in 25 minutes? Or if some are more difficult than others, could you complete all of them within 100 minutes? [Assume that you are seeing these questions for the first time.] If the answer is 'No', then the interview is too difficult. If the answer is 'Yes', then the interview is not too difficult.
  4. Alternatively, you could ask a former colleague or programming buddy to play the role of a prospective interviewee and see if they can complete your technical interview in a reasonable duration of time.
-1

One statistic is: how many fail? If you have had more than 10 fail what is the chance you really got 10 bad candidates?

Are they getting the answers wrong or just not finishing? In a strange environment 1 hour is not very long. I would only expect you could ask for like 4 basic tasks. In an hour you are only going to be testing some basic mechanics.

Maybe the questions are not clear and maybe the questions are too many / complex for 1 hour.

Could you post some of your questions?

  • Those are pretty basic tasks. I assume you give them access to documention. I would not have the syntax for all the datetime components memorized. I would have everyone have a boss even if they are their own boss. Looks like a reasonable test to me. Imagine you asked the to write a binary search – paparazzo Feb 24 '17 at 16:20
  • They have VS with IntelliSense, does that suffice? – Alexander Feb 24 '17 at 16:32
  • It was 4 candidates, btw. – Alexander Feb 24 '17 at 16:34
  • If you are expecting them to get timezone from culture that is kind petty. Then with the UTC time you break that. I am staring to question your understanding of DateTime in .NET. OK test but I think you make it more straight forward and still test basics. I would be asking myself why do I need a DateTime object. – paparazzo Feb 24 '17 at 16:39
  • @JanDoggen "For one statistics" alone is an answer. – paparazzo Feb 24 '17 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.