I flunked a technical test because some of their expected answers were wrong! Should I let the company know?

Question

A company I completed a technical test for sent me the list of their expected answers and why they thought my answers were wrong.

One of them was a SQL query which, if one were to implement it as they suggested, it would be much slower than my solution and would run the risk of blowing up with moderately higher volumes of data.

Another question was marked wrong because it didn't go as fast as their solution, but their solution only works if the data in the table doesn't ever change - which is unrealistic in production. Mine was slower because it would work no matter how the data was distributed.

The other answers they expected were very suspect as well in that they deducted a lot of points for nit-picky stuff that had nothing to do with my SQL skills. I suspect they had a junior person grade the exam.

Should I tell the company?

How should I handle my recruiter now thinking I lied about being a SQL expert?

Answer 1

You can push back via the recruiter, sure. The recruiter may or may not pass on your feedback, and it may or may not result in a different outcome.

Your feedback should be pretty polite though - so lose that "junior developer graded it" stuff, and explain in detail your answer to the SQL queries you mention.

You might want to ignore the nit-picky stuff unless you can politely word it, or politely word it as a learning opportunity for yourself.

---

I never said I was the authority on politely worded emails, but here's an example:

Hi Bob,

thanks for the feedback on my technical interview. It's obviously a disappointing result.

This is unusual, I know, but I'm really passionate about the goal of COMPANY, and I really liked the tech stack/team/environment/something so I was wondering if you would consider passing my own feedback about their technical test back to them? Hopefully this shows how eager I am to help COMPANY do whatever it does, or at least adds value to their processes.

In Question 1, I notice that the answer that they preferred would result in a runtime that is actually 25% slower than the one I presented, and would in fact even result in outages at larger data loads. This is because of SOME PRINCIPLE, and it's one of the areas I specialise in, and can really add value to help them grow.

For Question 2, the solution I provided was to handle changing data. Their preferred solution is faster, but only if their data never changes - otherwise their solution won't work. If they're using this code in production, they'll find a lot of value in my abilities as they scale up to handle constantly changing data sets.

If you think this will add value to Company, do please pass it on - as I said, I really love their MISSION/something and I feel that I can really provide value to them with my skillset.

Regardless, thanks for putting me forward for this role. All the best, Jack

Answer 2

This is one of those cases where you need to view the entire interview as a 2-way street.
You're interviewing them just as much as they're interviewing you.

If you'd really like to work here, then it may be worth your while to draft a response to the recruiter and interviewer indicating why you gave the answers you did and (gently) indicate why you believe that their expected responses are not optimal.

However, you should also consider the effort that they have (not) put into your interview and use it as an indicator of the environment you'd be joining if you were hired.

Answer 3

I'll add a different perspective. Maybe they sent you the results exactly because they wanted to elicit some sort of feedback from you.

Fully acknowledging that a written technical test is very much different from a face-to-face interview...

When I conduct technical interviews, on a couple of "how would you do X" type questions, when I receive correct answers, I would reply "Are you sure? I would do it this-and-this-way instead." I want to see how the candidate reacts - both technically and personally. This is especially important for senior roles. I want to ensure that the person can analyse their answer and act assertively upon confirming that the answer is correct. I also want to make sure that this assertiveness is still polite.

Think of it as a real-life test of "How would you convince somebody in position of power and authority that your opinion is correct - after they questioned it?"

Again, I fully agree that a written test is not the way nor place to use this approach.

Answer 4

I had a similar situation. The test was online. However, there were some mistakes. I took the test. Filled it with the proper answers (even if it meant failing). After that, I sent them a detailed e-mail, describing the problems I found in the test. They thanked me, and I have no idea what happened next, it is their problem.

---

BTW, clarification: I passed the test, but I took the risk of failing by filling in the correct data, not the "expected" data.

Answer 5

So, I have had experience with a decent looking company. I flunked their test due to poor explanation and very open-ended expectations. They had two parts, which was a coding section and the other was a find all the problems in this file. This was all in C, so it was painful as there are A LOT more mistakes that can be made and to be frank, I didn't know what all to look out for. They said both section should take only 2 hours in total, but you have unlimited time.

Failed the coding section, due to not understanding the problem well enough. There were a lot of issues with the explanation and it wasn't something that could simply be explained away. I had some errors in it, but no where near as big as the misunderstanding due to the poor examples and explanation.

Failed the analysis section, due to not finding all the errors they were looking for. A lot of the errors you'd have to be scanning this file like a hawk for and my theory was that going that far down was unnecessary if they were intending for you to finish that and the coding in 2 hours.

I got back to them and told them what I thought about it all, as kindly as I could. I had the realization after that about not wanting to use C because of how easy it is to shoot yourself in the foot and it will flatout let you do it. I explained that I misunderstood the problem and how it was supposed to work. Even put unit tests at the bottom of the file to test everything.

My advice is it's a good idea to give advice, just don't expect that it's going to change their decision. I considered it very kind of them to give me feedback, as it helps me get better at the risk of revealing more about the contents of their hiring test. So, I'd say if they had issues in it, then it'd be good to point them out but don't look at it as a chance to get yourself hired. Look at it as returning the favor for giving you feedback, which can be very rare nowadays. Besides, assuming everything you said is true, if they did that bad of a job of checking their content and aren't willing to admit their failure, then do you really want to work for them in the near future?

I'd let your recruiter know the problems as well, but you may not hear back from them for a while anyways. I've heard, but have not verified, that when applicants fail to get a job under a recruiter that they sorta ghost them about opportunities for a while as they believe that they won't get the next job either that they put the time and effort in to get them an interview for. They are paid only if you get the job, so it's just kinda the way it is. Not to mention, recruiters know the technical side at a very basic level, if at all, so they aren't likely to be able to understand the topic well enough to know what happened.

Answer 6

Same thing happened to me. Well, actually I refused to complete the test as I saw that not only were the questions ambiguous like that but they also involved too much actual work.

What I did was in the end equally work intensive. I produced a question by question critique of their test and sent it to the CTO with an explanation why I would no longer be proceeding with the interview. The CTO and hiring manager actually both called to thank me and I expect the dev responsible for producing the test got a rap on the knuckles. Basically I got through to further interview rounds and got the respect of the CTO.

One more thing: it is pretty unusual they send you test results. You might want to consider that they are expecting you to come back with feedback and that this is part of the test.