I'm currently applying to remote development positions and in every case I get past the initial screening interview, but bomb the automated coding test so am dropped out of the running before a technical interview.

These tests are written with web based IDE's and validated against (sometimes hidden) unit tests. These have many of the same issues as online homework in universities where a right answer isn't the right answer. For example $2.1 is the only correct answer even though it is less correct than $2.10.

As someone who is involved in hiring dev's, I get the point of tests. I've used them when hiring. But I don't let a computer grade them, I look for knowledge of algorithms and systems not trivia and syntax under time constraints.

The best analogy I can come up with is hiring a baker. They can describe how to bake a cake, they can use all the right ingredients and tools, but if they speak with a southern accent instead of a British one they fail. If they don't make it exactly the right shape (that hasnt been shared) they fail. If they take more than 15 minutes including going to the store to get ingredients, they fail. If they use flour instead of harvesting wheat by hand, they fail. If they ask questions they fail.

I can't fathom how this process results in useful employees.

So why is this so common, and how can a software developer who's good at software development and bad at pop quizzes get hired?

  • 1
    Ok come on. Off topic? > Finding employment (resumes/cv/cover letters, recruiters, hiring-managers, interviews, negotiations, etc.)
    – user29234
    Dec 22, 2019 at 21:47
  • 6
    A lot of the writing comes off as a bit of a rant, which is off topic. You also need to read the last sentence to know the problem you want to be solved. I did not vote to close though. Dec 22, 2019 at 21:49
  • 2
    "For example $2.1 is correct and is less correct than $2.10 which is not." - then report that as a bug, both to the company that's running the test and the company you're applying to. Neither of them are going to want to have bad tests.
    – Rup
    Dec 22, 2019 at 22:41
  • 1
    It's the time frame and subject matter. Answer a question in 3 minutes about a language feature released 18 hours ago. Clearly only those who studied the release notes pass.
    – user29234
    Dec 22, 2019 at 22:51
  • 2
    It was a php 7.4 question the day after release
    – user29234
    Dec 23, 2019 at 1:23

5 Answers 5


Why are software recruiters relying on automated testing of technical skills?

Because it's cheap. And it's fast.

  • 6
    And if you wanted to be extra pessimistic, it's likely recruiters lack the skills to do anything else. Dec 22, 2019 at 22:14
  • 1
    My assumption is that they are external recruiters who don't have wide language knowledge. Dec 22, 2019 at 22:25
  • 2
    To be clear Im only experiencing this from internal recruiting teams not professional headhunters
    – user29234
    Dec 22, 2019 at 22:30
  • Just like the companies using them :) Dec 26, 2019 at 9:05

The issue is that many people generally look for a way to beat the screening.

People lie on their resumes all the time, as that used to be one way to beat the system. And realistically, there is no way to catch most of those lies. A degree is easy to verify. That your code saved $100,000? Not really. As a result, recruiters are deluged in people who have great resumes but can't code FizzBuzz. Or more commonly, frontend/backend developers who claim to be full-stack. To solve this, they have turned to automated tests to try and weed out those people.

The problem is that those automated tests are not designed well, but the willingness of people to beat them remains high. A poorly designed test would normally just fail a ton of people and recruiters would become suspicious. What ends up happening is that people find a way to beat the test so that suspicion never happens.

I just graduated last year and lots of people at school had to do these kinds of tests to get jobs. How did they often do it? First, they had kamikaze candidates apply, i.e. people who were not interested in the job in question, but could qualify to get it. Once everyone got the bulk quiz offer, they piled into a room and worked on it together for the kamikaze candidate. 3 people or one challenge each. They would usually hit every test, so they just copied the code into their own applications, waited for the timer to hit something reasonable, and passed the tests.

There is a company I know that uses the same programming test year after year, semester after semester. They want to see a stack with the use of lambda functions and it auto checks for that. This test is also visually proctored via webcam. There is a solution people print off, stick to the side of their monitor, and make a fuzz about typing in slowly. It has yet to trigger any suspicion on their side that it is strange so many people easily pass.

You are not worthless, but your test-taking strategy is not on par with how a lot of people are likely doing it, making it seem like you are underperforming. There is a good chance that you are competing with people who already know the correct answer going into the test or are using a team of people to do the test.

I'm admittedly curious as to why a senior dev like you is having to deal with this. I hoped it would just be a junior-level thing. Are these jobs with Toptal or something?

Its testing a developers ability to pass tests. I can't fathom how this process results in useful employees.

HAHA, don't get me started on perverse incentives...

  • 1
    I've only seen this for remote work and even then only recently. Local employment in my area is more traditional hiring but is broken in other ways
    – user29234
    Dec 22, 2019 at 22:37
  • @evandentremont hiring will never be broken from the perspective of a recruiter because applicants will find ways. What is wrong locally? Dec 22, 2019 at 22:41
  • Employer culture is stuck in the 1800's along with the rest of the city. Measuring input not output, and paying absolute shit relative to the rest of the country.
    – user29234
    Dec 22, 2019 at 22:58
  • @evandentremont ah my fellow Canadian. You are in the Maritimes. That would explain why none of my uni friends went back there. Dec 22, 2019 at 23:10
  • That would be correct. Anyone sane moves.
    – user29234
    Dec 23, 2019 at 1:24

Because it's quick and easy. A lot of recruiters don't understand the technical side, and these automated tests are marketed with a promise to deliver quality developers.


When large and reputable companies post job offerings which target a widely available skillset, then they often receive a huge amount of applications. Far more than they can judge in a fair manner within a reasonable timeframe.

Interviewing everyone is just too expensive. They need some way to reduce the applicants to a manageable number. Preferably in some way which removes all the phonies who are so obviously under-qualified that they would just waste their time.

Automated tests are a great way to do that, because they scale so well. Once created, they can be administered to as many people as they want for virtually no additional cost. And they can get exactly the number of applicants they want by adjusting the score cutoff.

Automated tests are of course not failsafe. Sometimes the answers leak, and they have to deal with a couple posers. Sometimes the tests have issues which cause them to disqualify a couple candidates which might have gotten hired otherwise. But these drawbacks just outweigh the benefits.


I took one of these when I got my last position (well several with a bunch of companies but only one with the company I went with). After I signed on my manager explained that their main use of the tests was as a starting of point for discussion, meaning that it complemented the CV. Having been aiding a bit in recuting I also feel that they are a great source of questions for an interview.