A colleague of mine came to me for advice. He wanted one of those high dollar jobs at a well-known MegaCorp. He knew a leetcode test was used by them. I suggested that his top priority was to study the leetcode books for the next 3 weeks. I told him those three weeks would be the best ROI he'd ever make.
He did so. He aced the test. He got the job with a huge offer. He agreed that the ROI was out of the park!
P.S. I'm curious what your test is for obvious frauds?
Let's assume that instead of leetcode, the company had decided that the qualifying test for programmers would be sketching a vase of flowers (about as relevant as leetcode for commercial programming).
Your friend would have spent three weeks practicing their sketching, right? And they would have made a decent drawing at the interview. And that would have the exact same ROI as spending those weeks on practicing leetcode.
The point is not that studying leetcode is good ROI. The point is that large orgs are testing for something that isn't relevant to commercial coding.
This is classic Streetlight Effect [0]. Just because it's easy to test leetcode, and it appears to have something vaguely to do with coding, it gets used.
Like I said in other posts, I would use the "pick a bug and fix it" method. But I'd also screen down from 100 candidates to ~20 or so before getting into technical interviews.
And, again, I don't think leetcode will scale well to 100 interviews either. You need technical staff in that interview room, and if you subject your team to 100 technical interviews using leetcode they're not going to be happy about it.
Doesn't this imply that, had your friend not taken your advice, he would have failed and thus been an "obvious fraud", but a mere 3 weeks transformed him from fraud to competent programmer?
If a good candidate can do it in 3 weeks, then a middling one can do it in 10 weeks, and a bad one in 6 months of memorizing. They all pass the screen, but two are lower-quality, and the bad candidate is about as much of a waste of the "real" interview as a total fraud would be.
If the bar is that low, just use an accredited degree, certification, references, or a proctored quiz with questions like "define a linked list in < 3 sentences," and it would be just as effective at filtering the very lowest, while more pleasant for everyone involved and maybe cheaper. Crammable puzzles that don't represent real software engineering don't provide much more value than those filters would.
> they likely don't have the ambition to get the high pressure jobs.
What absolute drivel. I've done plenty of leetcodes. I don't care about writing a qsort algorithm, I know the trade-offs, unless you're working for a FAANG or FAANG-adjacent company, the need to write your own sorting algorithm implementation is probably zero or near zero.
That wasn't the point. Would you wash your car before picking up your date for the first time? A dirty car would drive just as well, but your date will figure you don't care about the date going well.
It's like cramming for an exam. Yes you can get enough information in your short term memory to pass, even do well. But a year later (or a month) it's all gone. You never really knew it.
These arguments all sound familiar to the "high stakes testing" debate about the SATs. The tests were unfair, did not measure college readiness, could be gamed, etc.
Then universities dropped the SAT requirements. Even MIT was suckered into dropping them. It was a disaster. It turns out the SATs were a very strong predictor of college success. MIT and others reinstated the SATs.
Ways SAT is more reasonable than typical leetcode interview:
- you can take it ~3 times and most colleges will give you the best score without penalty
- you can use your result to apply as many times as you want
- if you feel stuck you can still get a good score by moving on to the other problems
- it's administered uniformly across the world
- you don't need to narrate your thinking while you're still processing information
- you get an objective score instead of a pass/fail that may be subjective
All true, but still the arguments presented here sound the same as the arguments against SATs and "high stakes testing". I checked a bunch of major software companies, and all of them used leetcode as a screening device. (Google, Microsoft, Nvidia, Meta, Apple, Anthropic)
We pull in everyone from the Team who is available at interview time into a room with a big a$$ whiteboard and we talked to the candidate. We work through the problem, either current one we are working on or recent one that we completed or we have few "golden ones" we like to have a chat about. Afterwards the Team decides whether or not they want to continue solving problems with this candidate or not.
Exactly what we do on day-to-day basis is what we do when we are trying to grow the team. I would guess that 85% of my team would fail leetcode-style screening right now (which is good cause it is garbage)
All this proves is that BigCo puts a lot of value in these idiotic leetcode tests. It doesn’t imply at all that it gets rid of “obvious frauds”, and in fact seems to suggest the opposite given that someone was able to “ace” the test just by buying a book and studying it for three weeks.
> He did so. He aced the test. He got the job with a huge offer. He agreed that the ROI was out of the park!
...and everybody clapped?
I have news for you: passing an online leetcode is now so easy that an AI can do it. If you think you're catching the cheaters, you're just wrong.
Leetcode was always a stupid test of competence, but it was cheap and it had a high true-negative rate, which filtered the riff-raff. That isn't true anymore.
Sure, sure. So now we have the worst of all possible worlds: an expensive interview process where the candidate has to be onsite, and a stupid evaluation mechanism that doesn't at all reflect real-world job requirements. And in exchange for the inconvenience of an in-person interview, the candidate gets treated like a number and is met with a stupid memorization gauntlet that tests a skill that quite literally will never be used again now that we have AI. Sounds great.
Man, there are times when I really hate this industry.
The only job application process I've ever walked out on in the middle of was 25 years ago, when I was taking a proctored computer skills test and it dinged me for pressing the Windows key rather than moving the cursor to the Start button and clicking it.
So this is a new presentation of an old problem, I guess is the takeaway there.
> an expensive interview process where the candidate has to be onsite
I'm sorry, but if I'm doing the hiring, the person who isn't willing to come on site to do an interview will be one of the first people off the list to hire (I mean, unless it is a remote job, and the person is nowhere near the employer, obvious exceptions would apply).
In-person interviewing can be extremely valuable. Will it filter out every bad candidate? No, some people excel at faking it. However it can be a big help in finding a good candidate.
> I'm sorry, but if I'm doing the hiring, the person who isn't willing to come on site to do an interview will be one of the first people off the list to hire
Nobody said anything about not coming on site. That's how we used to do it, in the ancient pre-history of 2019. What I said was that if you bring someone on site only to leetcode them, you deserve not to hire anyone.
The downside, of course, is that you don't have the cheap, dumb pre-filter of a memorization test anymore, so you'll have to figure out a better way to winnow down the applicant pool whom you're willing to pay to bring on site.
Maybe finally software engineering will achieve a level of interview maturity seen in other professional fields...but I doubt it.
So if it’s just a shit-test to see if the applicant has enough grit to grind through an arbitrary filter, and not to measure their aptitude, why not use Minecraft or something? Come into our office and show us how long it takes you to kill the End Dragon.
I took an online technical interview recently where there was a check-box they made you click before every section, promising you would not use AI to solve the test for you.
After they rejected me I had to point out that their test is selecting for the cheaters.
I would view skeptically any testee who turned in a perfect score.
When you check in at the airport, the machine asks you to certify you are not carrying banned items. Of course, one wanting to break the law and carry banned items would certify it. The purpose is that if you get caught with the banned items, and you certified you did not, it's an extra charge they can paste you with.
Once we carried dangerous (not banned) items and selected yes and the first action by the attendant was to come over and say we selected yes by mistake and she selected no for us
Honestly this is purely because time is money. Its a blunt instrument that is surprisingly effective. Unfortunately, it also removes a fair amount of competent people too, that simply don't think/operate in a leetcode mindset.
if you need garbage like leetcode to filter out "obvious frauds" you got a whole lot of problems in your company/team/...