40

I've recently attended an interview for a software developer position. At one point I was given three questions - two code writing and another more theoretical. One of the code writing questions included writing a parser for arithmetic expressions string. I've came up with a working algorithm, but I didn't have time to write the code (also, there was another more efficient algorithm which the interviewer told me about later).

At home I've checked the algorithm - Shunting Yard), which is quite complicated (my algorithm was even more complicated and was similar to Operator-precedence algorithm). Is it realistic to expect an applicant to implement this argument in 20 minutes on paper without Internet access and answer two more moderately hard questions? What were the expectations?

1

11 Answers 11

95

I find these sorts of tests are most suited to recent graduates who remember their theory from university. When a company asks me to do a test, I usually decline and tell them to hire a graduate instead as I would most likely fail the test since what they are testing for are things I haven't used in over 15 years. It causes some friction with recruiters but that is not my problem.

I'm happy to have a whiteboard discuss-as-you-go type technical interview for a company which extends me the courtesy of giving me their time to interview me. Any company asking for a coding test before even wanting to meet me gets a hard pass.

My advice is don't get too hung up on it, written tests before meeting a candidate is just lazy interviewing.

To further elaborate: When I try to explain this to recruiters as to why I decline such tests I use the analogy of the English language. We all use the language daily, we're good at it (some better than others). But I bet I could stump you (the recruiter) with a grammar test for something you haven't revisited since school and to top it off I bet I could find obscure words in a dictionary that you don't know. Then I can cheerfully declare that you the recruiter, a native English speaker, doesn't know how to speak English and hire a school leaver who does remember the obscure and mostly unused/irrelevant details.

19
  • 47
    I completely disagree with the characterization of written tests before meeting a candidate as "lazy interviewing." It takes quite a bit of time to set a proper test up and evaluate it. It is also a waste of time and money to interview people who aren't minimally competent at writing working software, which is a surprisingly large number of job applicants.
    – Joe
    Feb 28, 2023 at 10:11
  • 21
    @Joe I understand that opinion when it really is a "proper" test (well thought out and role-appropriate), but it seems far more common to just buy in a generic test from a 3rd party, resulting in the problems mentioned in this answer.
    – DBS
    Feb 28, 2023 at 11:26
  • 25
    I would never hire anyone without a test first. I've been burned by fake CVs too often and a test, even a simple one, is remarkably effective at weeding out the really bad candidates so I don't waste my time, or theirs for that matter, interviewing people who don't have the basic skills required. Just because you have had bad experience with some tests is no reason to generalize as you do in this answer. And when I have around 100 CVs to look through, I need some way to quickly whittle the list down to a manageable level.
    – terdon
    Feb 28, 2023 at 11:29
  • 9
    "I would most likely fail the test since what they are testing for are things I haven't used in over 15 years" this rather misunderstands how you reasonably use good coding tests (admittedly lots of companies don't understand this either). It's not really about acing the test, it's about figuring out how a developer approaches a new problem. Eric Lippert had an interesting post on this years ago that really influenced my thinking on this.
    – Voo
    Feb 28, 2023 at 13:09
  • 11
    @Joe most companies just use 3rd party tests. The person looking at the results is often someone from HR who will just use whatever grading system the testing company reports. This is lazy interviewing. My resume and work history speaks volumes, I dont need someone to see if I can write a recursive search algorithm in under 15 minutes that works to judge me, and yes I would probably fail considering I haven't written one since uni. Maybe I am a bad software engineer. Hire a graduate, they'll ace those sorts of tests.
    – solarflare
    Feb 28, 2023 at 21:23
85

We don't know what the expectations were. Maybe they wanted to filter the 0.1% of people, according to their definition of 0.1%.

Maybe they wanted to see how you approach hard problems under stress, and actually solving this wasn't the purpose. Maybe they just wanted to see how far you got.

Different companies have different hiring criteria. If you fit them, you get the offer. If not, search for another company.

If the company regularly filters out so many people that they find nobody, they will have to rethink their hiring approach.

11
  • 4
    It's probably worth clarifying that if they wanted "to filter the 0.1% of people" this is precisely not how to do it Feb 28, 2023 at 10:31
  • 9
    @ScottishTapWater some people have interesting ideas what 0.1% constitutes and what skills are important and what aren't. So yeah, this was a guess as to their motivation, not a recipe how to effectively find the 0.1%
    – Benjamin
    Feb 28, 2023 at 10:42
  • 4
    Maybe "some 0.1%" (not "the best 0.1%") is enough if they are getting too many applications. Mar 1, 2023 at 0:26
  • 3
    I suspect the 0.1% figure is hyperbole. US colleges graduate about 8000 aerospace engineers per year. About 1/4 of them are trained in the field that my current employer works in. Finding the top 0.1% (as if that's possible) would mean two job offers per year, and those two people will of course receive multiple job offers; we won't get both of them. When it comes to programming, the fizzbuzz test comes to mind. (This is no longer used as a filter as it is too widely published.) The purpose was to filter out the 60% to 90% of comp sci graduates who can't program. Mar 1, 2023 at 11:02
  • 4
    @Nelson Not long, about a year. The 'hard test' was the whim of the founder, and the company had serious founder syndrome. In hindsight I'd say it was an amber flag, not enough to walk away at that point, but it does form part of a picture looking back. Mar 1, 2023 at 18:24
25

This isn't reasonable from the company, nor is it helpful.

I'm a technical lead for a big software company and do plenty of recruitment. I've worked across a number of companies, and none of them do the whole "invert a binary tree on paper" thing that some big American companies seem to do.

We don't do it for a few reasons:

  1. It tells us precisely nothing about a candidate's ability to do the job. You're, quite literally, never going to have to write code on paper and there are very, very few situations you wouldn't have access to Google (we freely tell candidates to use Google during their tech tests if needs be).
  2. Unless you're writing framework, or very specialist code, a good 90% of a software engineer's job is stitching together various disparate systems to get them to work coherently. Writing readable, testable, and maintainable code that does this reliably is often far more important than pulling the most optimal algorithm out of your ass.
  3. Finally, it adds totally undue stress and pressure on the candidate. You're making them do something that they'd never normally do, in a way that they wouldn't normally do it, with potentially hundreds of thousands of whatever your local currency is on the line. That's horrible. Why would you do that to someone and what impression does that give them of your company? It would almost be acceptable if you got some useful information out of it, but you don't. Your employees shouldn't be working in an atmosphere of undue pressure and fear, so why does it matter if your candidates can perform under those conditions? It doesn't.

So, my advice, although I understand it might be somewhat difficult to follow at your stage in your career:

  1. Refuse to engage with this crap. If someone tells you to write code on paper, say no. If, as an industry, we stop engaging with this, it will stop happening.
  2. Go and find better companies that actually understand what makes a software engineer good at their job and go work for them instead. You'll be far happier for it.
  3. Any company that engages in gimmicky "lateral thinking" or Myers Briggs style interview stages, isn't worth working for. If they don't care about the efficacy of their recruitment pipeline, then they don't care about their employees.

And remember, you work in software engineering. We're basically the most in demand profession in the world right now, and that's becoming increasingly true as time goes on. There are far more jobs than there are competent software engineers, we have all the power. Interviews are two way streets, if companies don't treat you well during them, go and find another company. Also, make sure you feed back to them exactly why.

Then hopefully we can stop this stuff from becoming even more prevalent.

10
  • 5
    While I understand the sentiment (I never ask for written tests in interviews for much the same reasons you state) I find the final suggestion (just say no and walk away, and point-blank suggestions that the company is not worth working for) not particularly helpful. There is a middle ground here; OP could simply do their best in the time alotted (and coming up with the algorithm as they did, even if in informal form, would be great IMO!). Also, the HR of a company may be very disjunct from their actual workplace athmosphere, especially if an external headhunter is involved...
    – AnoE
    Feb 28, 2023 at 13:44
  • 11
    I've done plenty of recruitment also, and the biggest problem I have is that a frightening majority of candidates cannot actually write code. But that's what I'm hiring them to do! I don't want someone who can maybe cobble something together, that works by accident, after asking for help on Stack Overflow. I want someone who can write code, so I generally ask that it be done in the interview. I give plenty of leeway for syntax errors. But if that's too much pressure for a candidate, that's too bad, because, you know, when deadlines are looming we have to program under pressure, too. Feb 28, 2023 at 14:26
  • 19
    @SteveSummit - To be clear... I'm not saying don't make them write code during interviews... We make them write code during interviews... We just have them do a pair programming exercise on a fairly representative task with one of our engineers. Crucially, on a computer, in an IDE, with access to google if they need it. That way we can see they can code in fairly realistic conditions, we can also see how well they can collaborate on a problem and explain and discuss what they're doing. That's far, far more useful to us, and far, far fairer on the candidate Feb 28, 2023 at 14:32
  • 1
    I think it depends on the test. A team I was working with was interviewing for a QA, a position that required writing a LOT of SQL queries. One applicant had extensive SQL experience on their application, but could not even write a simple select statement in the interview. Feb 28, 2023 at 16:16
  • 2
    If it's boiling down to that, you should be able to establish that fairly easy by talking about it rather than worrying if they can write it down on paper... "Could you talk me through how you'd write a SQL query to get XYZ out of this database?"... Their ability to remember the exact way to do a left outer join on the spot is somewhat irrelevant if they know the concepts and can quickly google the syntax. I do take your point though if it's that trivial Feb 28, 2023 at 16:42
20

Recruiters sometimes like to ask candidates to write code on paper to assess their problem solving skills and ability to think for themselves.

It's also worth noting that recruiters may deliberately choose questions that are more challenging than what you're likely to encounter at work. They want to see how you deal with difficult problems and whether you have the skills and knowledge needed to work on complex projects.

In your case, it looks like you did your best and managed to come up with a working algorithm, which is a good sign. Even if your algorithm wasn't that efficient, the fact that you find a solution shows your problem-solving skills and willingness to tackle difficult problems.

Or.. sometimes the project burns and they are looking for someone who will solve current problems in the best possible way.

3
  • We "write code on paper" in our interviews, but the "code" is pseudcode or whatever the interviewee wants to write it in. It is about the thinking process, the rest can be trained.
    – Tony Ennis
    Mar 1, 2023 at 19:33
  • 1
    @TonyEnnis Yes, but why? Many programmers or engineers don't remember everything and have to google the simplest things. For example, the difference between slice and split in JS ;) Mar 1, 2023 at 20:52
  • You gotta have 2 neurons to rub together, in some places!
    – Tony Ennis
    Mar 1, 2023 at 21:15
13

Most interviewing questions has both an overt and a subvert component.

By asking you to produce some code, they're determining whether you can write code. That's the surface part.

Underneath, they're perhaps gauging how you interpreted the instructions, whether you missed anything that was given in the instructions, how you interacted when perhaps clarifying a deliberately vague point (immediately, after starting, ignoring the vague thing, or being oblivious to the vagueness)

They might also be checking if your indentation style matches what they were taught in school, or if you use CamelCase variable names, or other stylistic niceties that don't affect the function.

Simply do the best you can, and it can help to talk out loud more if you're coding on a whiteboard. This gives a view into your thought-processes which is really what the questions are trying to show.


Side story - I remember doing this verbalisation years ago while taking a drivers license practical test. Instructor said at the end it was very helpful for him, he knew I'd observed things, even if I hadn't needed to react. I also made a point of turning the head and looking at things rather than just one's eyes.

The point here is to show you know what you're doing, and doing it the "right" way. At the end, the code you wrote is a means to an end, which is demonstrating You.

2
  • 2
    That's very true. Some places have an entire "evaluation guide" to go along with the test that's available to interviewers but not to the candidate. It may list the skills that are supposed to be tested by each question, or the typical or mind boggling answers to prepare to encounter, or especially any follow-up questions to use if the candidate aces or deflects the primary question, thus avoiding the subvert component (temporarily). Mar 1, 2023 at 16:29
  • 1
    Side story - I failed my first drivers license practical test attempt by NOT doing that verbalization and that ostentatious head gymnastics and the reason given to me was correlated to that. Something about not yielding to imaginary traffic correctly. Mar 1, 2023 at 16:37
7

Another option is that your interviewers didn’t think through their question or thought the answer would be easy and obvious. Don’t assume that they’ve had dozens of interviews and streamlined the whole process.

I’ve had colleagues come to me with “We have an interview this afternoon for a position which is somewhat related to yours, do you have a coding example or some questions we could give the candidate from your field?” And I’d suggest some things but without any idea about the experience of the candidate or how much time they’d be given. We don’t have a lot of interviews, especially not for some particular positions.

6

One option that I haven't seen mentioned is that it's possible you misunderstood the question, and thus overcomplicated it for yourself. Either through your own fault of reading in things that the question never explicitly asked, or through the question writer's fault for accidentally asking for something they never actually intended candidates to answer. For instance, perhaps they only intended for candidates to solve the problem for a certain subset of arithmetic expressions, or ignoring rules of operator precedence.

Quite often interview questions are written with deliberately vague requirements — candidates are then rewarded for asking clarifying questions. While personally I prefer to ask questions that are clear in their core requirements and maybe have some interesting edge cases which I hope candidates will notice and ask about, it's probably a good lesson to learn to always ask clarifying questions, especially if at first glance a problem seems impossibly hard. This reflects well against the real world, where sometimes client requirements are much more limited and narrow than software engineers are prone to naturally assuming.

So my advice is, next time you're in an interview, ask plenty of questions. If you're worried about coming across as clueless, you could phrase your questions in more leading ways (eg "should I assume that I need to follow operator precedence rules?") to make it clear you've already considered a possible answer and just want confirmation. If the company you're interviewing with reacts badly to this, or doesn't have an interview framework where you're able to ask these questions in the first place, then I'd suggest it's not a place any software engineer should want to work as you'll probably be spending most of your time implementing things your clients don't even want...

5

It's possible that the interviewer was checking to see how you'd respond to being given a task with an impossible time frame. In my opinion, the best answer would have been to say "I don't believe I can complete this task in the 20 minutes allowed". That tells the interviewer that you're not shy to raise alarm bells when you've been set an unreasonable deadline, and that's actually a huge positive in an employee.

In response, the interviewer might then have said something like "just complete as much of it as you can", or maybe "well then, estimate how long it would take", or even "that's the answer I wanted, let's move on to the next question".

In one interview a few years ago, I was given a small Java programming task and told to complete it in 10 minutes, on paper. Instead of completing the task, I wrote a JUnit test for the task. This impressed the interviewer, and I got the job. It was kind of a calculated risk though, and you may not feel brave enough to try this.

1
  • The benefit of this approach is that if the interviewer is unsatisfied by such an approach in this stressful situation, then maybe that what it is like to work there. Interviews are 2-way and such interactions can tell you that this isn't the job for you.
    – Tony Ennis
    Mar 1, 2023 at 19:31
2

Job interview always work in 2 directions.

You don't know for sure what are theirs expectations but you should know yours. If you feel under pressure, or otherwise uncomfortable on the interview, there is a great chance you'd feel the same when working for them, because the interview is a part of the working culture in a more way a company would want to.

I suppose the hiring manager found it cool to test how the candidates perform under pressure and unrealistic deadlines, which means, that both of them are likely to be part of their everyday business. Now, would you like to work for a company like that? If the answer is 'yes', because you need any job hardly, then hope for luck, because such tests are often luck-based (you get correct idea by chance, while 10 other candidates won't). If not, just move on.

1

Twenty minutes is not enough time to write this program. I assume more time was given for the other program and the theoretical question.

They're just trying to see how far you get and find out what your thinking is. If you get flustered and you can't explain your approach, that tells them something. If you had a great approach but just couldn't finish it, that tells them something too.

Another possibility is that the interviewers were simply inexperienced.

0

In my opinion, giving a task that cannot reasonably be written in the time specified, especially if it can only really be written in that time assuming no mistakes. However, as a caveat to that, when I worked at Google, if you were doing one of the coding interviews, you were supposed to have a goal question that would take the entire half hour of the coding part of that hour interview slot. The thing is, we didn't expect most people to hit that, and we would have earlier, and easier, versions that we start off with. For example, one question I was fond of involved converting a binary tree to an XML document, and the base case allowed them to restrict the input to, say, standard strings with no spaces, and to not worry about recursion, degenerate trees, and the like. Once they solved that (which took most candidates about 10-15 minutes), we could start dipping into more complicated scenarios and how they might handle them. I only had a handful of candidates who got the hard version of the question coded in the time specified, and those were the ones who got marked as potentially exceptional (only potentially, of course, because some people came in having trained on particular questions that they'd seen on coding sites as Google interview questions).

And, at that, we were explicitly told that the ultimate correctness of the answer wasn't nearly as important as the approach of the candidate (whether they asked clarifying questions, how they communicated how they were going about the task and what they were having trouble with, etc) and their ability to either keep moving forward, or indicate what they were stuck on. Engineers being engineers, not everyone was able to keep to that, but I can remember several cases where the candidate didn't get the right answer, but we could tell their approach was sound, and in at least one case I remember, they got the nod to move forward in part because they recognized that they'd gone wrong, and much of the rest of that interview was them explaining how they could have been testing during implementation to catch it in the future.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .