12

I am seeing so many companies ask LeetCode questions for software engineering jobs. (even for just developing website's front end).

Those questions can be very specific and not something that a typical software engineer would do even in 15 or 20 years. For example, it could be, given 10000 x 10000 slots and dividers of varying height between them, now find out a 20 x 20 region that can trap the most rain water when it rains for a long time.

And there can be 1500 different questions in the LeetCode database.

And the catch is, many programmers can write some code that can find the answer, but the problem usually has some tricks in its problem's property that, somehow you may be able to do it faster.

And researches were done for these types of problems, sometimes for months or for years, as a master or Ph.D. thesis.

But the thing is, programmers, especially programmers who develop the front end of a website, don't do it at all. Not even once in 20 years. So it is not really their real job content.

But the companies like to ask you these questions, and people who read the answers on LeetCode, they may get 5 out of 5 points for the several interview sessions and get into the company, while the person who wrote a correct solution but not the solution that took people months or years to find out before, get a 3 or 3.5 out of 5 only, and they are rejected as a candidate.

And what I found sometimes is: these engineers enter the company, and sometimes they cannot even write correct code. They know standard answers to those questions, but when seeing a real problem, they did not know how to think sometimes -- not as well as the candidates who can actually think well in the interviews.

But companies hire people using this method any way. I do not quite understand how and why it works this way?

1
  • my friends said he was asked an impossible question and he directly walked out of the interview, but then he worked at this trashy Android testing company and it got acquired and he got acquired into Google, and then since then he claimed "Oh these Leetcode questions are so fair." Talk about slave type of thinking. Nov 19, 2022 at 0:49

9 Answers 9

29

I do not quite understand how and why it works this way?

One very important thing to keep in mind is that the goal of a company's hiring process is to hire qualified people, as easily as possible. That's it.

Its goal isn't to make sure that every qualified person gets an interview. The goal isn't to find that "diamond-in-the-rough" candidate, someone with a poor employment history, or who does poorly interviewing, or who forgets things while white-board coding, or who makes mistakes on a timed coding assessment, etc., but would actually succeed at the job.

As long as the jobs are being filled by good hires, the hiring process is working. Are there "false negatives", folks being rejected by the process who are qualified, and who would have succeeded at the job? Of course there are, but from the point of view of the hiring process, that is necessary. A company doesn't have the time to interview every person who applies, so it needs some way to filter applicants so that the folks who actually do get interviewed are very likely to be qualified. If they interview 5 qualified people for a role, it doesn't matter that 5 other qualified people didn't make the cut.

Note that I'm not claiming that LeetCode type questions make the best assessment of a developer, or that a company which uses them thinks that they do either. What I'm saying is that it doesn't matter one bit if these questions have zero relevance to the job. It doesn't matter that you're never going to reverse a string, or find all the prime numbers in a list; all that matters is that the company thinks that its process is working: that it is finding qualified candidates as efficiently as possible.

So if you want to apply to a company that has decided that they are going to use these tests, you have two choices: you can either practice these problems, or you can sit down unprepared and hope you get lucky.

7
  • 10
    Also worth noting that false negatives for the company are far less expensive than false positives. You'd much rather reject a few people who would've done a good job so long as you (almost) never hire someone who's actively bad at it. Leetcode may not be great at that, but they do at least weed out people who can't program.
    – Kaz
    Nov 13, 2022 at 0:36
  • 1
    @Kaz that's total BS... you ever read the comic about a Ph.D. went into an interview and the 23 year old asked him "how do you swap two variables without a temporary one. Quick, in 30 seconds". The Ph.D. couldn't give the "standard answer", as it is meaningless and nobody actually does it in practice, and he got rejected. And so, "he cannot program"? The Leetcode is also full of these "you know the answer if you know it, otherwise it takes 5 years to figure it out" type of questions, so people memorize them, if you didn't memorize, you don't know how to program? Jan 29, 2023 at 10:08
  • 13
    @StefanieGauss true, but that's exactly the point this answer + Kaz were making: they don't care much they rejected that Ph.D. (a false negative), as long as they got enough qualified developers without many false positives. Kaz didn't say "everyone weeded out => can't program", he said "can't program => will be weeded out". Nov 30, 2023 at 16:41
  • 6
    I heard a story of how a manager reduces the candidate pile. He takes half and throws it in the trash. Why? Because he doesn't want to hire unlucky people.
    – Nelson
    Dec 4, 2023 at 0:59
  • I will note that certain types of tests will weed out good candidates with surprising reliability, so the point you make has some caveats: Really good candidates might not bother with such a process in the first place, because they don't have to. So it's not as simple as you make it out to be. I've heard of companies complaining they can't get good candidates, but then ask people to jump through N hoops to get hired.
    – bytepusher
    Dec 4, 2023 at 2:36
25

But companies hire people using this method any way. I do not quite understand how and why it works this way?

Some companies do not have experienced engineers analysing the interview process. A failing on their part if they're hiring engineers, but common enough. So HR just searches for some metric to use.

12
  • 6
    @user10186832 I am not a developer, but I have seen an engineering question used where the expected answer was wrong and explained why. And I have seen a compliance question where the expected answer had been illegal for a couple of years since a legislation modification.
    – Kilisi
    Mar 27, 2021 at 11:11
  • 3
    @TomTom I find that people who claim that 90% does X do not fully understand the subject Mar 27, 2021 at 18:47
  • 1
    Yes, may be a software think. Such as the whole question IS ABOUT IT. Or you think they make those coding tests with floor people in retail? And it is not locale specific - seen this in multiple countries, always larger more established companies where IT is a "side story" (i.e. their business is not making software).
    – TomTom
    Mar 27, 2021 at 23:30
  • 1
    Sounds like it's just a cultural thing. I've had exactly 0% of the companies I've interviewed with use any of these tests. It's a big world and you probably only interviewed in a small part of it.
    – Erik
    Mar 28, 2021 at 12:37
  • 5
    Large software companies like Amazon ask leetcode-style questions during their interviews. The interviewers are engineers or engineering managers themselves. I doubt that these companies have a lack of engineers reviewing the hiring process.
    – zmike
    Nov 12, 2022 at 22:55
10

From the point of view of an interviewer: A considerable amount can be learned about a coder's competence from watching how they attack the problem, what assumptions they make, what questions they realize they should ask, and so on. That often doesn't even require that the candidate provide a complete solution. It may feel ridiculous, but there actually is value when it's done right.

This doesn't mean all interviewers are trained to do it this way; some really are just flailing.

5
  • Only if the interviewer know something about IT (or at least the specific argument), else how I attack the problem and what assumption I make are meaningless since you cannot understand if they are valid in the first place.
    – Gianluca
    Nov 14, 2022 at 14:46
  • 1
    In most cases, you are being interviewed by the department that you might be hired into -- so their competence to evaluate you should be assumed.
    – keshlam
    Nov 14, 2022 at 16:02
  • I am not sure if the "standard way" or SOP is to say "our expected answer (that he should not have known unless if he has seen the answer before): 5 points", and "great answer but not that standard answer above: 3 or 4 point", so the great programmer is out and the memorizers are in Jan 29, 2023 at 10:13
  • I don't think most of the folks using these would penalize an equally good but different answer. They might ask you to explain it's advantages and disadvantages over other possible solutions. BUT. I'm adding a competent hiring process; if you really want to worry, I can't prove that you will always be wrong.
    – keshlam
    Jan 29, 2023 at 16:24
  • 1
    @StefanieGauss I've interviewed with a couple companies that did ask some sort of what you probably would consider leetcode questions - none of them wanted the perfect text book answer and for none of them someone just memorizing it would have gotten the job (as far as I can tell). For most companies these questions are a conversation starter to get interactive with the candidate on some made up problem. The problem doesn't matter much but how you try to solve it and discuss it. Same as any technique there certainly are some companies that misapply it in some way (or spin it differently). Dec 4, 2023 at 19:43
9

Because HR and recruiters quite often are totally detached from reality (as in: delusional of their worth) and do not know anything about IT. They look for SOME metric to measure how good people are without bothering the IT department with a trial workday or something like that.

And that means they need measurable tests. And then you add them and possibly some manager being so stupid (yes, that is the friendly version) that he does not realize that EVERY IT person I have ever met realizes those tests mean nothing - that they promptly use a totally inaccurate test.

This is seriously a case of mass delusion - a whole industry thinking they mean something when everyone doing programming knows they mean nothing.

It gets even better when then those incompetent hires do not get fired again because "it is so hard to get ANYONE from our HR department, we better work with what we have". That then means the Peter Principle is in full effect, the IT group becoming a molasse of incompetent people (as the competent ones will move on), all because HR does a wonderful job with totally non working tests.

1
  • 3
    Remember that some of those managers got there by the Dilbert Principle.
    – Skrrp
    Mar 27, 2021 at 22:22
2

Hiring is a time consuming and costly process. Many companies offload that on to recruiters and/or online tech tests.

These style tests are very well marketed.. They claim to provide companies with an almost haste free way to vet candidates, ask the right questions, find the best talent, and that's why companies use them.

If you don't like doing them or don't see how they are relevant, then say so. Ask for a different way to prove your skills. - It doesn't always work, but sometimes it does.

Those questions can be very specific and not something that a typical software engineer would do even in 15 or 20 years.

These style questions are 100% relevant to software engineering and what a "typical" software engineer should know, but completely irrelevant to a lot of niche areas within software engineering such as web development because most of the CS knowledge is abstracted away by high level languages and libraries.

Most of these style tests are focused on software engineering as a whole, not on nice areas, so a lot of companies are giving out CS tests, not web development ones.

(Rather the web development focused ones are often poorly written with a focus on irrelevant CS questions)

Some companies are starting to realise this, but it will take time for things to change.

Essentially companies just want the most economical way to find good candidates, and these style tests promise that, even if they don't provide.

3
  • 3
    "These style questions are 100% relevant to software engineering" - the questions are nieche. I do software for 30 years, backend systems and db specialist. This is what the VAST majority of companies program - systems that take data from a database and play around with it in a UI or processes. And then you get hit with low level questions and asked to program a SORT? Heck, I do not sort in code - I tell the database to sort the data. Yes, I can look it up, but this is low level code that is hardly ever written in the real world, and if then in VERY specific special contexts.
    – TomTom
    Mar 27, 2021 at 23:32
  • right. One time I even studied Quicksort and Mergesort well like its implementation which I never do in 20 years, just for interviews. And upon seeing I was familiar with how to implement them, the interviewer asked, "so, are they stable?" which actually means, if the comparison is equal, are they same order in, same order out? In real life, I never ever care about this (especially I am in front end), but he asked this question for what? Oh by the way, that company has 95% of employees the same ethnicity... so... maybe those questions were designed to reject, rather than to hire Jan 29, 2023 at 10:17
  • @op: let me guess, are you a recruiter?
    – mrodo
    Dec 5, 2023 at 9:48
1

If you have lots of candidates then spending time on each one gets very expensive. So anything that filters 500 candidates down to 10 is good for the company. It's also valuable if you can't be accused of discrimination when you removed 492 candidates. So a "leetcode" question is quite good in this respect.

You can be a good developer without being able to solve that kind of problem, but the ability gives a small hint.

1
  • So, use an AI driven interview. Done.
    – TomTom
    Dec 5, 2023 at 9:51
0

My company does a similar style of questions and I think it is a good way to filter out developers who know how to solve a general algorithmic problem.

For example, it could be, given 10000 x 10000 slots and dividers of varying height between them, now find out a 20 x 20 region that can trap the most rain water when it rains for a long time.

Since this question is incomplete, I will make some assumptions here, but typically with a task like that you can observe the candidate doing the following steps, in about 45 minutes:

  • Clarify the question and assert any assumptions made.
  • Model the problem
  • Come up with an algorithmic solution
  • Implement it in a common programming language
  • Explain the solution to peers
  • React to feedback and adjust the solution

This of course assumes that the candidate works in front of the interviewers and explains their thinking process. All the points above are relevant to the day to day work of a developer.

And the catch is, many programmers can write some code that can find the answer, but the problem usually has some tricks in its problem's property that, somehow you may be able to do it faster.

Usually the "trick" in the questions we ask is something simple as sorting the input before applying the solution to turn a N^2 into an N log N solution, or something like that. A developer experienced in writing efficient code, usually can solve it or will get to the solution after some pointers.

One might argue that this problems don't come every day, but only one inefficient loop written per month, can already make your code's performance considerably worse.

So while this style of questions might not be perfect, it is pretty good in assessing a wide area of skills needed in a software engineer, and so far we have not found anything better.

-1

Partly, there are historical reasons.

The archetypal question of this kind is reversing a linked list. At the time when it was first used for interviews, a lot of programming work involved slinging around pointers and linked data structures, so it was immediately relevant to the job. It was an excellent interview question: one which a competent programmer could do in their sleep, which would stump an incompetent programmer, and which could be communicated to the candidate in a few seconds.

Over the decades since, the work and the interviews diverged.

On the work side, linked data structures got wrapped into standard libraries or even into the languages themselves. Even if the language still has pointers, in most everyday use they're just opaque handles, passed around but not really rearranged in any substantial way. Actually working with pointers and linked data structures is now somewhat of a specialty.

On the interview side, the questions got more esoteric as people started cramming the usual ones, or simply by a process of elaboration.

A couple of decades later, we're at a situation that's the equivalent of interviewing bike couriers with BMX tricks; a related skill, to be sure, but not really relevant to the job.

Not much a candidate can do about that, other than practicing the BMX tricks.

-8

I want to add an answer here, which is to claim, it is not like the company doesn't know what they are doing. They know exactly what they are doing: they want to hire the obedient type, not the smart type.

This kind of pattern repeats in history. 1000 years or 2000 years ago, if you can recite and memorize some ancient script or text, you become the official of the dynasty and in a way, rule other people. The people who did not bother to memorize 200,000 words of scripts, will be ruled by the people who are willing to be obedient to the Dynasty instead. Is that reasonable? Maybe not.

However, if it is not reasonable, will you do it? Will you be obedient enough to do what is unreasonable to obey us? If you obey us, we will have you rule other people. Otherwise, not. The "obeying" is the key here.

The same with education back in Hong Kong or perhaps in UK some years ago. (I am not sure about nowadays). The physics or advanced math they taught you, was not for you to understand. They tell you how to solve something, memorize some equations, for example, the length of a string, and the weight of a lead ball, and now move it 45 degree, and what is its vibration frequency? If you memorize the formula in an exam and give the answer, you get 10/10 points. Otherwise, you get 0/10 points. The ones that can get 95/100 points, gets to go to a university. (Only about 2% of the Hong Kong population could get into a university at that time, because Hong Kong at that time only had only two universities for a population of 6 million people). And when you get into a university, you will become a lead or engineer in a company and get HK$8000 a month. Otherwise, you go in with a high school diploma, and you are a "technician", and get perhaps HK$3000 a month. A lot of people could not afford to study overseas, so it was not like "oh just go to a college" like in the US. (but even in US, look at what happens when you are willing to do Leetcode or when you are not willing to regurgitate optimal solutions).

So memorizing formulas, is that reasonable? Perhaps not. But are you obedient enough to do that? Hong Kong especially had this thing about colonialism. Which is: UK wanted to rule HK (as a colony). It could not use UK people to directly rule Hong Kong people, as people would not like it. So they pay ridiculously high salary to Hong Kong people, say, when they are senior, HK$15,000 every month, so that they obey the government. And then, getting such high salary, they will in turn become the government officials and rule the people. They tend to follow orders because they don't want to lose their job and get into the private sector and perhaps earn only $11,000 per month instead. And so the key point is, the government wanted obedient people. Obedience is the key. As a result, they give you an education system that is what local people called "stuffing a peking duck" and a lot of people thought it was unreasonable, and most everybody were following that system. You see, if you obey, you will be high up in society. If you don't obey, they'd make sure you stay at the bottom.

3
  • 3
    I never did any leet code questions, but probably I am asking similar style of questions in Programming interviews. I doubt that memorizing 1500 answers is actually the easiest way to get good at this.
    – Helena
    Nov 12, 2022 at 12:03
  • 4
    I did not downvote. But, I don't think solving Leetcode means that you are obedient to the boss or company. People can't memorize the solutions to all 1500 Leetcode problems. The best way is to understand the logic of each solution. Leetcode has its own pros and cons. Some Leetcode problems are great as they help people to write efficient code while other Leetcode problems are just fancy/useless puzzles. I agree Leetcode is not the best or most accurate way to measure the skills, talent, or intelligence of a programmer. Hopefully, many companies will improve the hiring process. Nov 13, 2022 at 21:04
  • 1
    no, you didn't get it. It is the "willingness" to use Leetcode that shows your obedience. Just like when you don't understand physics but is willing to memorize formulas and willing to "calculate" something you don't really understand Nov 16, 2022 at 4:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .