104

A company I completed a technical test for sent me the list of their expected answers and why they thought my answers were wrong.

One of them was a SQL query which, if one were to implement it as they suggested, it would be much slower than my solution and would run the risk of blowing up with moderately higher volumes of data.

Another question was marked wrong because it didn't go as fast as their solution, but their solution only works if the data in the table doesn't ever change - which is unrealistic in production. Mine was slower because it would work no matter how the data was distributed.

The other answers they expected were very suspect as well in that they deducted a lot of points for nit-picky stuff that had nothing to do with my SQL skills. I suspect they had a junior person grade the exam.

Should I tell the company?

How should I handle my recruiter now thinking I lied about being a SQL expert?

  • 1
    Comments are not for extended discussion; this conversation has been moved to chat. – Lilienthal Feb 20 at 8:08
  • 2
    A note here is that we're taking the OP's word for it that his answers are "more correct" and the the test/tester is "incompetent" (to use a term bandied around in answers and comments). We should entertain the possibility that the OP could be wrong or that we're missing important information since we don't know the questions, or the correct/OP answers. (No offense JackP I'm not saying you are wrong only that we cannot be certain) – Mr. Boy Feb 20 at 9:54
249

You can push back via the recruiter, sure. The recruiter may or may not pass on your feedback, and it may or may not result in a different outcome.

Your feedback should be pretty polite though - so lose that "junior developer graded it" stuff, and explain in detail your answer to the SQL queries you mention.

You might want to ignore the nit-picky stuff unless you can politely word it, or politely word it as a learning opportunity for yourself.


I never said I was the authority on politely worded emails, but here's an example:

Hi Bob,

thanks for the feedback on my technical interview. It's obviously a disappointing result.

This is unusual, I know, but I'm really passionate about the goal of COMPANY, and I really liked the tech stack/team/environment/something so I was wondering if you would consider passing my own feedback about their technical test back to them? Hopefully this shows how eager I am to help COMPANY do whatever it does, or at least adds value to their processes.

In Question 1, I notice that the answer that they preferred would result in a runtime that is actually 25% slower than the one I presented, and would in fact even result in outages at larger data loads. This is because of SOME PRINCIPLE, and it's one of the areas I specialise in, and can really add value to help them grow.

For Question 2, the solution I provided was to handle changing data. Their preferred solution is faster, but only if their data never changes - otherwise their solution won't work. If they're using this code in production, they'll find a lot of value in my abilities as they scale up to handle constantly changing data sets.

If you think this will add value to Company, do please pass it on - as I said, I really love their MISSION/something and I feel that I can really provide value to them with my skillset.

Regardless, thanks for putting me forward for this role. All the best, Jack

| improve this answer | |
  • 2
    I think it is definitely good to send them detailed well written notes and I would do this directly to the company or ideally the person who you met with if at all possible. You never know they might be impressed. It's also a helpful thing to do for future applicants – Mr. Boy Feb 19 at 11:47
  • 58
    I definitely agree with losing the cutting tone. However, I would also dial back the whole "I love the company/environment/mission" tone, as it strikes me as disingenuous coming from somebody who never even worked there. – Bridgeburners Feb 19 at 13:24
  • 3
    @Bridgeburners well, you can love a company mission - maybe they help sick children. Or maybe you're really excited about working in finance (or whatever) because it is an exciting environment filled with bold, smart people. It's not hard to think of something flattering and nice to say about a company and the people that work there. – bharal Feb 19 at 16:09
  • 2
    @bharal - I aspire to someday have the professional communication chops you demonstrated in this answer. You're an inspiration. Do you do commissions? :) – William Feb 19 at 16:42
  • 2
    A very good answer. To the point, although the OP maybe able to fill a dozen pages with various issues, the message will be stronger if only the top two or three are mentioned. This shows that the sender can prioritize, values people's time, and isn't motivated (solely) by a desire to show off their own skills; an extremely valuable business communication skill. – CCTO Feb 19 at 17:22
87

This is one of those cases where you need to view the entire interview as a 2-way street.
You're interviewing them just as much as they're interviewing you.

If you'd really like to work here, then it may be worth your while to draft a response to the recruiter and interviewer indicating why you gave the answers you did and (gently) indicate why you believe that their expected responses are not optimal.

However, you should also consider the effort that they have (not) put into your interview and use it as an indicator of the environment you'd be joining if you were hired.

| improve this answer | |
  • 2
    And would you want to work for a company who didn't want to consider other solutions to their textbook ones? – Liath Feb 19 at 11:33
  • 4
    @Liath the people you'd be working with and the person in charge of tests might be different so it's hard to make such a snap decision I'd say – Mr. Boy Feb 19 at 11:46
  • 6
    True, but company culture is pervasive! – Liath Feb 19 at 13:12
  • 3
    @Mr.Boy Agreed, but if they've put someone incompetent in charge of the tests then that might say something about the company as a whole (assuming OP's analysis is correct). – Jon Bentley Feb 19 at 23:01
  • 1
    @Tomáš Sure, but if they've put in effort in the wrong place (not assigning someone well qualified enough to judge the "wrong" answers on the test), that can be just as bad as not putting in the effort at all. – cjs Feb 20 at 1:54
8

I had a similar situation. The test was online. However, there were some mistakes. I took the test. Filled it with the proper answers (even if it meant failing). After that, I sent them a detailed e-mail, describing the problems I found in the test. They thanked me, and I have no idea what happened next, it is their problem.


BTW, clarification: I passed the test, but I took the risk of failing by filling in the correct data, not the "expected" data.

| improve this answer | |
  • 3
    If it was a test, how did you know what the expected data was, let alone that it was wrong? – Aleks G Feb 19 at 13:59
  • 3
    It was several years ago, I do not remember the details. Either it was a multiple choice test and the correct answer was not there, or I found the problems when I got the results - question to self: did I receive the results of the test?! Sorry that I cannot be more helpful. – virolino Feb 19 at 14:02
8

I'll add a different perspective. Maybe they sent you the results exactly because they wanted to elicit some sort of feedback from you.

Fully acknowledging that a written technical test is very much different from a face-to-face interview...

When I conduct technical interviews, on a couple of "how would you do X" type questions, when I receive correct answers, I would reply "Are you sure? I would do it this-and-this-way instead." I want to see how the candidate reacts - both technically and personally. This is especially important for senior roles. I want to ensure that the person can analyse their answer and act assertively upon confirming that the answer is correct. I also want to make sure that this assertiveness is still polite.

Think of it as a real-life test of "How would you convince somebody in position of power and authority that your opinion is correct - after they questioned it?"

Again, I fully agree that a written test is not the way nor place to use this approach.

| improve this answer | |
  • That's a great technique! I wonder how many interviewers do something like this – Vivek Chavda Feb 19 at 18:59
  • 1
    It is a great technique -- it very effectively lets the interviewee know which companies think that lying and manipulating are a good interview strategies and totally not representative of how they conduct business. Seriously, you can probe and test and elicit thorough responses without playing headgames. – Matthew Read Feb 20 at 3:59
  • 3
    @MatthewRead Asking someone to stand by their opinion in the face of opposition isn't playing headgames it is a good interview technique. For my current role the interviewing manager questioned my solution and presented his alternative, I was able to correctly identify a potential bug with his approach and he even admitted to having that occur on his first attempt. I firmly believe this is part of why I got the job. Had they not asked this question I wouldn't have been able to demonstrate that. – linksassin Feb 20 at 5:27
4

So, I have had experience with a decent looking company. I flunked their test due to poor explanation and very open-ended expectations. They had two parts, which was a coding section and the other was a find all the problems in this file. This was all in C, so it was painful as there are A LOT more mistakes that can be made and to be frank, I didn't know what all to look out for. They said both section should take only 2 hours in total, but you have unlimited time.

Failed the coding section, due to not understanding the problem well enough. There were a lot of issues with the explanation and it wasn't something that could simply be explained away. I had some errors in it, but no where near as big as the misunderstanding due to the poor examples and explanation.

Failed the analysis section, due to not finding all the errors they were looking for. A lot of the errors you'd have to be scanning this file like a hawk for and my theory was that going that far down was unnecessary if they were intending for you to finish that and the coding in 2 hours.

I got back to them and told them what I thought about it all, as kindly as I could. I had the realization after that about not wanting to use C because of how easy it is to shoot yourself in the foot and it will flatout let you do it. I explained that I misunderstood the problem and how it was supposed to work. Even put unit tests at the bottom of the file to test everything.

My advice is it's a good idea to give advice, just don't expect that it's going to change their decision. I considered it very kind of them to give me feedback, as it helps me get better at the risk of revealing more about the contents of their hiring test. So, I'd say if they had issues in it, then it'd be good to point them out but don't look at it as a chance to get yourself hired. Look at it as returning the favor for giving you feedback, which can be very rare nowadays. Besides, assuming everything you said is true, if they did that bad of a job of checking their content and aren't willing to admit their failure, then do you really want to work for them in the near future?

I'd let your recruiter know the problems as well, but you may not hear back from them for a while anyways. I've heard, but have not verified, that when applicants fail to get a job under a recruiter that they sorta ghost them about opportunities for a while as they believe that they won't get the next job either that they put the time and effort in to get them an interview for. They are paid only if you get the job, so it's just kinda the way it is. Not to mention, recruiters know the technical side at a very basic level, if at all, so they aren't likely to be able to understand the topic well enough to know what happened.

| improve this answer | |
  • 2
    Thanks for taking the time to write such a complete and well thought out answer. I'm not really all that keen on getting hired by them, to be honest. My motivation was courtesy, but also some professional pride. Even if they change nothing, at least they'll know I didn't actually fail their test and that they may have some gaps in their knowledge they need to fill. – JackP Feb 18 at 23:01
  • 3
    "I've heard, but have not verified, that when applicants fail to get a job under a recruiter that they sorta ghost them about opportunities for a while" – In my experience working with recruiters, if I fail to get one job, then the recruiter continues to work with me more on other opportunities often than not. There's no guarantee, of course; a recruiting agency's job is to find candidates for the given role, not to find roles for the given candidate. – Tanner Swett Feb 19 at 4:13
  • 2
    "Failed the coding section, due to not understanding the problem well enough." coding is not everything. In real life, sometimes, understanding a problem is the biggest part. And this can be part of the test as well; How good is the person on finding out what the real problem is? – roel Feb 19 at 8:25
  • 1
    @roel I agree with your point and I probably wouldn't be so salty about it, but they encouraged you to rush by saying how long it should take and saying they could see when you started work on it. They used a very poor example for input, where output didn't change at all as the only example of output for one of the cases. This case also included little to no explanation. They specifically set you up to fail by encouraging you to rush and providing just enough information to appear like you're on the right track, but not enough to know the true intention. They got me, but it was a shitty test. – Shirohige6969 Feb 19 at 15:01
  • 1
    @roel To make matters worse, it easily would've taken an expert C developer more than the time described to complete the task. They probably would've been able to find all the errors within 30 minutes to an hour, but due to the complexity of the coding problem, it would've likely taken another 2-3 hours. It was simple enough, but in C where there are no String objects it required many additional layers of logic for this to work. With the additional curve balls they had thrown in to the incomplete question, even someone who would've asked would've wasted a good amount of additional time on it. – Shirohige6969 Feb 19 at 15:06
-1

Same thing happened to me. Well, actually I refused to complete the test as I saw that not only were the questions ambiguous like that but they also involved too much actual work.

What I did was in the end equally work intensive. I produced a question by question critique of their test and sent it to the CTO with an explanation why I would no longer be proceeding with the interview. The CTO and hiring manager actually both called to thank me and I expect the dev responsible for producing the test got a rap on the knuckles. Basically I got through to further interview rounds and got the respect of the CTO.

One more thing: it is pretty unusual they send you test results. You might want to consider that they are expecting you to come back with feedback and that this is part of the test.

| improve this answer | |

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .