One of the problems I've seen constantly, for 10+ years (been working for 20+, interviewing others for approx. 10+) is that there has never been a "baseline" skill level that you could always count on when you got a candidate in front of you. Any given person's programming and problem solving skill level could be absolutely anywhere on the spectrum between 1. "super proficient, fluent in multiple languages, highly structured thinking, clearly articulates a solution, can mentally visualize over a range of 20 orders of magnitude" and 2. "literally has no idea how to solve a problem, program a computer to do it, or even describe coherently what they are thinking". And these two extremes can have the same resume.
Some first level of the hiring funnel (HR, recruiter, or something) screens the candidate by looking at the resume and maybe doing an initial phone screen, and then this person gets dumped on the interviewer, and they 100% could be anywhere on that skill scale. So you always seem to have to start at "how do you turn a computer on?" and "write code that finds a substring in a string." It's really frustrating for the interviewers and I'm sure also insulting to actually good candidates.
I've never been in one of these businesses, but I bet if a hospital interviews a doctor, they can at least count on basic knowledge of anatomy, and if a law firm interviews a lawyer, they can assume the person has read a law book. We have no such guarantees in software.
The amount of time and energy wasted due to recruiters and hr is so frustrating for me, Ive solved live coding exercise so fast that people thought i was cheating, solved tests in a fraction of the time, etc, overall I can easily do tech stuff in interviews, and just be dumped because I am asked if I am willing to go to friday drinks or company parties and I say I don’t really want, yes, asking if one has the will to go to company parties is so important that its part of the interview process, this system is broken
> Ive solved live coding exercise so fast that people thought i was cheating
I've once been rejected in an interview because I found a solution that the interviewer didn't think would work, spent half an hour trying to prove me wrong rather than trying to expend the smallest effort to understand - even after I proved it was correct. I was quite glad they showed me a bit of the culture though
I've had one interview with as a PHP/Symfony developer somewhere around 6-7 years ago, where I architected the classes in the app to receive dependencies as argument rather than querying them through the container, and was suggested because it was not best practices, and a couple of years later Symfony itself re-architected dependencies management the way I did in that test, some people shouldn't really be allowed to interview, most recently I've had this tech lead telling me he was architecting an innovative health monitoring system for their app, and was running every 10-15 minutes UNIT tests in the production app, asked why UNIT test would suddenly fail in production found out his UNIT tests were not UNIT, just integration querying actual resources
"Unit" and "Integration" (and E2E) are the most bullshit "technical" terms we have to deal with on a regular basis as an industry. At best, they can only tell you that integration tests are on average more liberal in their use of system calls than unit tests, that is if the codebase in question even has both.
While this dev you encountered might have had no idea what they were doing, a most charitable explanation is that they may subscribe to the idea that the "unit" is a business usecase within a single service (often a 1:1 correspondence with an endpoint, but sometimes even more than one, e.g. a form upload with a separate signed upload step to a 3rd party service).
FWIW the concept of small/medium/large as presented in the Google testing blog is a much more useful denomination.
That is one interpretation. An interpretation that, once we start thinking beyond the first concept that comes to each particular mind, ends up just as meaningless:
What is a component? A global variable, a pure function, a procedure, an endpoint, a business usecase, a microservice, a microservice fleet?
What is the whole, if not the same list of questions just asked in backwards order?
It gets worse: Once you arbitrarily come up with one definition that you stick with in your project, does the definition you chose leave little to no room for interpretation? Do all kinds of tests that are worth writing fit neatly within those definitions?
Technical terms that are so loosely defined we can do without, for only two things ultimately result from their use: Apathy, or madness.
You have to be willing to game the system juuuust a little bit. "Yeah, sure, sounds fun". Then don't go. If your actual coworkers instead of the HR drone appreciate you, noone will care. If they do give you shit for "not being a team player", go find another job. But in my experience the former situation will be much more common. You just gotta get past the braindead HR stuff.
If someone in an interview basically says they just want to be left alone to program (or whatever), that's at least starting to send a red flag up the flagpole. I'm sure there are some of those at many large companies but the most influential developers I've know were pretty good at working with people and casually interacting.
I'm very active in companies chat, always step up to help when I have a solution, always step up first when I notice something can be improved, I am not sure how you went from not be willing to socialise/be drunk/be around drunk people, to "be left alone"/"not able to casually interact"
On the other hand, there are companies where devs are expected to wait for a Jira ticket to be assigned to them, code it, test it, merge it, goto 1. That's the reality of the job whatever they might claim externally.
"I just want to be left alone to program" seems like a perfect fit: will do the job, will not ask for anything.
In that case--although it won't happen--I'd expect to be told that I'll be a cog who will be dropped or replaced at any moment. Perhaps a contractor position.
I have no social anxiety, at my last company I was praised by manager because I’d be the one asking questions and keep discussions going without suffering from timidity, anxiety etc., I just don’t care
Programming inside a business is more about going to company parties than programming fast. And I prefer people who don’t program fast, but that program for the future of the business. You might be better suited for a different career shape like foss maintainer or small, highly technical companies
I don't have anything to say about the programming fast, because that's an assumption, I've had problem solving solutions fast, but overall I spend the time to make sure the app is tested and maintainable, but why should tech people care about company parties, what value does it add for the company? I'd like to understand that because I usually adopt ideas I agree with when I am explained, last time was "To make people connect to facilitate conflict resolution" as if when I have conflicts with people I don't know I shoot them
I dont think there is a single answer like "facilitate conflict resolution", it's more of a matter of generally fitting in the corporate world, being able to build relations in that world, and not give the interviewer confirmation about its bias against "huge nerds that ace every technical challenge but then don't shower and can't talk to people without snarky remarks". yes, the bias is alive and well, and in general programming is a communications job (we communicate with stakeholders and then communicate with machines, but instead of studying marketing we study programming languages)
but let's not derail, "no i dont want to go to company parties" = "doesnt want to socialize" - "doesnt want to submit to the company" - "will probably be a hassle to work with" - "can't communicate" - and so on.
again, think hard about a different career angle in tech if you dislike these things, you might enjoy it more
Getting to know your coworkers in more social environments eases interactions with them in the workplace. As a senior/staff level engineer working with and coordinating across teams, socializing to build repertoire and ease these interactions pays dividends quickly. HR asking if you'll go to company parties is their indirect way of probing to see if you could handle this aspect to fit a senior role, but it's not imo the best way to find out if someone can handle this. As someone who doesn't drink alcohol, I totally understand where you're coming from not wanting to go to parties or bars after work. I try to socialize before and after meetings with coworkers, during lunch breaks, etc. outside of party/drinking scenarios.
> I'm sure also insulting to actually good candidates
I don't mind it much. I don't have great social skills but I don't mind answering simple questions. If someone is insulted by simple questions, they're probably gonna be a bad mentor to junior devs, right?
I ask very simple question as a build-up for harder questions (write a function to reverse a single-linked list), probably half of the candidates fail to do in any reasonable time.
I feel like it's quite quick actually. I think of software eng interviews as binary search type of thing. You ask a mid difficulty question. If they don't know answer to that, you'll ask 0.25 difficulty question and so on. Although obviously, if the first question was a fluke, you'll adjust your cursor, so in the end it's not entirely binary search, but it is similar in its approach. Perhaps there is a better analogous algorithm for that.
The point is that, if they can answer very well more than 75% of your questions, you increase the difficulty, otherwise you decrease.
And the better they answer, the more you try to challenge and argue them.
What I do hate however is leetcode. I'll avoid that completely as an interviewer, as I feel like it would only help me determine whether they are the type of people to spend hours on leetcode.
I'll want to try to understand their passion to build and produce a product that solves something and provides value for their customers. Leetcode culture is contrary to that to me. To me leetcode culture is something where you play the game to increase somewhat irrelevant skills to get to a prestigious company to optimise for your total compensation. Which is fine, but it's not something I think that would be great for me to try to find best members for my team.
I haven't looked for a job in a while, but I'm bringing this up because in my experience when I was the candidate I remember people being quite rigid in their questions, for example questions being predetermined, which meant they ask me non-sensically easy questions, which considering what kind of questions I had answered before it would make no sense that I wouldn't know answer to those questions. So a lot of time-wasting.
Or in the other direction they could be asking similar questions in similar difficulty and area of topic, where someone clearly shows they no experience with, instead of trying to understand all talents of the candidate.
To be fair, it's our own damn fault. "Poor stewardship over professional standards" is a succinct way of putting it.
The reasoning comes down to poor training, lack of standard competency tests (which is a stable goal to train for), and probably heavy disagreements about what a candidate should be capable of as a junior/mid/senior/principal, or at year 1, 2, 3, 5, etc. of their career.
Communication of these standards over blog posts and what you can find via search engines just won't - and hasn't - cut it for raising the competency of the industry as a whole. If you are competent, you can publish a book which becomes popular, but there would still be disagreement about what books are important.
As an example, Designing Data Intensive Applications is probably one of the more well-known books, but it's not a requirement to read that. The reason it's not a requirement is that you can meander through most jobs by just googling stuff and no one denies you a job if you can't talk about its content.
This problem has gone on long enough that most of us can't give a comprehensive list of what the "basic knowledge" is that we need. Most of us could try to make our own lists, but we'd be much less confident that that list should be the standard because it hasn't received any critical review and approval by a large sample of people.
We need to get out of this free-for-all thunderdome. We do that by getting the companies aligned on what they expect. Then they simply don't hire people that don't meet those expectations. Then we can start writing competency tests towards meeting those expectations and simultaneously develop training material to pass those tests. Most importantly, those standards cannot change too quickly otherwise we end up with the same moving target situation we have now.
You give me the training material, I pass a test, then I get a junior job and continue training for the expectations at various job title levels. It should be that simple.
In the case of the lawyer, if it's a big city firm, (whether this is right or not) they'll look at where they went to school, whether they were on law review, and who they clerked for. And with that done, it doesn't matter a lot anyway because they'll be an overworked associate who can be given lots of crap work to do and if they don't make partner, they'll probably be pushed out.
Funnily enough, one of my best jobs in tech included an interview with who would become my hiring manager. (I don't think the company had actually figured that out at the time as they were writing the job description for me.) I don't think I had ever had an interview that I just couldn't read like that one. Years later, after he was no longer with the company, I sort of asked him if I had really bombed and he basically laughed and said he was always tough on interviewees. (And he was always really tough to read.) It also probably didn't help that I hadn't had a real interview in at least 10 years.
> In the case of the lawyer, if it's a big city firm, (whether this is right or not) they'll look at where they went to school, whether they were on law review, and who they clerked for.
This, plus grades and class rank. And there is a series of interviews, obviously.
For a young associate, grades/class rank are a decent proxy for how hard working they are and whether they have an aptitude for the basic practice of law.
> I've never been in one of these businesses, but I bet if a hospital interviews a doctor, they can at least count on basic knowledge of anatomy, and if a law firm interviews a lawyer, they can assume the person has read a law book. We have no such guarantees in software.
If the cost of those guarantees is formal specialized training costing hundreds of thousands of dollars, I'll stick with the world of phone screens - speaking as someone who's been both rejected-after-phone-screen and rejected others after a phone screen.
I used to feel the same way, and yes, reliance on credentials and formal training often serves the credentialing and training industries. But, I don't think our industry's current 'free for all' ultimately serves anyone.
We should consider an inexpensive, very basic "programmer's bar exam" to establish a (admittedly very low) proficiency baseline. I'm not asking for a test that gatekeeps software engineering only to people who can write a compiler. I'm asking for a reliable document that says this person can write a loop and a conditional statement and can reason about how it works. Is this a lot to ask for?
TBH, I don't really get the whole "insulting to good candidates" idea.* People who have been in industry long enough should understand that this is pretty common. If they get insulted, that's probably bad signal since they are taking it personally.
On the other hand, if the person is annoyed or frustrated by the idea I don't blame them.
* Caveat for people with high levels of experience going into more senior job titles.
Fizz-buzz tried to set this, but I’ve run into people who have trouble making compound decisions with compound data, so I ask slightly tougher ones, like sort these users by name. First, last, middle, maybe age or date created.
Or take this array of items and give me a map that uses the name as a key. I had one guy who could do a lot of front end work well but he could not pivot data to save his life. It was super weird. Like Derek Zoolander ambi-turner weird.
There's nothing weird about it. There are many programming domains where maps and pivot data are totally irrelevant. They just never come up. Developers who have specialized in those fields for years might just need a quick refresher on data structures.
It was a situation where someone had to fix his code three times in six months. There’s a point you sit down and sort it out. Ajax data doesn’t always match the page structure.
I've worked with a couple folks from MIT - which I imagine is "top N" for a pretty small value of N for most anyone - who knew a lot but could build very little quickly or effectively.
What you get graded on in schools for CS or even SWE is pretty different from what you get graded on on-the-job.
CS/EE degrees aren't particularly focused on programming so if you want to hire programmers then that would be a rather silly filtering criteria. It's like if I need to hire a plumber I don't look for candidates with chemical engineering degrees.
This is a very contentious statement. I studied CS at two very different EU universities and there was a massive focus on theory plus programming. All courses had huge coding projects where we had to build real end applications, including courses with a very strong theoretical component, which made things really tough.
I suspect real truth OP is not sharing is that they are quick to fire. If this is default in their company, then it’s ok to pick up incapable coder, since it will be corrected in a month or two.
Simple coding questions are invaluable at filtering, but of course do not guarantee the result. In last 3 weeks I interviewed 3 candidates, two of them failed to write 40 lines of code based on written requirements. One of them even used Copilot.
While I have never been in sales, the saying I've always heard is that "sales managers have no trouble firing people." There's a very quantitative output--your numbers--and, if you don't make them, even for reasons that may be out of your control--well...
Why does a doctor or lawyer get a pass? Plenty of admins etc are just cramming exams as well, I'm sure there are plenty of doctors out there that are as bad as some of the programmers/admins out there.
yes agree and .. in the mid 2000s there was a concerted double-edged change in interviewing that I could see. One came from Redmond and was an exercise in intimidating the candidate with a whiteboard, and the second came from Google where they interview you like you just defended your PhD at a top University.. so do it again now with the problem of my choice. Like any trend, the third, fourth and fifth level players heard of this and imitated it.
All of this combined with seniority not for coding, but for the mercantile aspects - if you formed and sold a company then you are senior, otherwise no.
I'd probably fail the author's interview. As soon as you hit strong cultural disagreement (e.g. "new technologies" is a minefield), the interviewer has to be a mental Hulk to proceed without bias. Goodbye diversity.
I've also done 100s of interviews (small and large companies). What I learnt:
- CVs are useless except for leveling the candidate (which is done for me),
- CVs are dangerous because they create a bias ("looks like an amazing candidate!" -- an then you get someone who is good, but not amazing as you expected and then you rate them lower than you should -- I intentionally skip the CVs as much as possible),
- majority of the people will solve a simple problem at the algorithmic level, and then fail to code it up,
- the best value is to give the candidate a SIMPLE coding question which produces a decent amount of code (25-50 lines), and observe them while solving it (their approach, bugs, etc.),
- the signals I'm looking for: (1) can the candidate code fluently, (2) do they approach the problem in a systematic way, (3) can I communicate with them without friction.
> As soon as you hit strong cultural disagreement (e.g. "new technologies" is a minefield), the interviewer has to be a mental Hulk to proceed without bias. Goodbye diversity.
I think this is unavoidable. Teams have cultures (I don't mean ethnicities, though of course there is some overlap), and ill-fitting candidates will struggle even if they are hired. Or they'll change the team and stress someone else out. As un-PC as it sounds, probably small, values-aligned teams are the easiest/most comfortable to work in, even if that means individuals aren't exposed to more diversity.
> the signals I'm looking for: (1) can the candidate code fluently, (2) do they approach the problem in a systematic way, (3) can I communicate with them without friction.
Isn't this a sort of cultural preference on its own? You're looking for (and prioritizing) people who can code a certain way and communicate with you a certain way. A brilliant programmer from a different code style background, who's maybe an introverted poor communicator (or just have a different communication style) and codes in nonstandard (to you) way might not pass such an interview. That's fine... they probably wouldn't work well with your team anyway. But IMO that's the point of cultural fit interviews, to respect both your team and the candidate's time upfront by evaluating not just their technical ability but how well they'll fit into your team.
The "signals" you're using are just a masked variant of that. Maybe (as an example, just an extrapolation) you prefer async communications with clear, simple assignments -- like self-contained Jira tickets -- rather than group architecting, pair programming, or whatever. But those are cultural values.
If you ignore all those signs and forcibly include someone, sure, you might increase diversity on your team... and stress for everyone, including the new hire who finds themselves alienated from the rest.
>> the signals I'm looking for: (1) can the candidate code fluently, (2) do they approach the problem in a systematic way, (3) can I communicate with them without friction.
> Isn't this a sort of cultural preference on its own? You're looking for (and prioritizing) people who can code a certain way and communicate with you a certain way.
Fluent coding doesn't imply style or something non-measurable. You can say if someone is fluent or not even if they are using a programming language that you don't know. I interpret it as: do they struggle while converting their idea to code, or not. Not how they structure the code, etc.
Same for the systematic approach to the problem. If their explanation is all over the place, or failing to uncover the important bits of the problem[1], bias doesn't kick in here. We can argue about the appropriate layering of the solution (the level of detail). That's where different people are different. But you can see if their strategy works or not once they put it in code (i.e. maybe they should have put more structure in their approach).
[1] (I: "Code me a string invertor". C: "Yes Sir! Here you are." I: "That's not what I had in mind.").
> A brilliant programmer from a different code style background, who's maybe an introverted poor communicator (or just have a different communication style)
Well, if I can't communicate with them (there's high friction), I shouldn't work with them. This is a bad fit by definition. They might find a better fit at another company.
> As soon as you hit strong cultural disagreement (e.g. "new technologies" is a minefield)
I am not sure what you mean by this. This is a question I ask and I’m concerned there’s something I’m missing here. Could you elaborate?
> I intentionally skip the CVs as much as possible),
That seems very odd. How do you comb through and make an initial choice of who to interview then? Also, seems unfair to not look at the work they’ve done over all the years.
Personally, I’m a GitHub person. Show me what you’ve written and how active you are.
>> As soon as you hit strong cultural disagreement (e.g. "new technologies" is a minefield)
> I am not sure what you mean by this. This is a question I ask and I’m concerned there’s something I’m missing here. Could you elaborate?
Easy :). Take me: I hate Go and think Java is the best programming language there is. (My daily drivers today are C++ and Python.) NodeJs should've never been invented. I'm pretty opinionated when it comes to this. I'd still accept the position where I have to code in Go (been there and it was OK for both sides), but I wouldn't keep my mouth shut during the interview -- I wonder how that would go with 80% interviewers.
> How do you comb through and make an initial choice of who to interview then?
Sorry, I didn't put it explicit enough. I don't do the initial selection or leveling. This is done for me. The CVs are useful for this. But if you can avoid reading the CV (e.g. assessing only the raw skills), then you should.
I feel like 99% of eng interview advice sounds just like this: "I've interviewed a bazillion people, everyone else is doing it wrong, here's the one good way."
Honestly, I think a lot of it is akin to medieval quackery. "I've treated hundreds of patients with leeches and the large majority of them eventually recovered - you should listen to me!"
I wish that we, as an industry, would spend less time reinventing interviewing from first principles. There are decades of research on how to interview people effectively. I highly recommend Schmidt + Hunter's meta-analysis (https://www.researchgate.net/publication/232564809_The_Valid...) as a starting point.
From my read of the research:
1. Use work-sample tests. Yes, they're annoying, but they're high-signal. Take-home tests, online tests, in-person tests, trial periods - there are plenty of options here, but it seems clear this is the most effective way.
2. Use structured interviews. Engineers love to just have loose conversations and judge people based on that, but structured > unstructured. Plus (just my guess here), I suspect unstructured interviews leave a lot more room for bias.
3. Plenty of techniques are effective on their own (eg references), but add minimal effectiveness when you're already doing a more effective test (eg work sample) and are therefore pretty much wastes.
>1. Use work-sample tests. Yes, they're annoying, but they're high-signal. Take-home tests, online tests, in-person tests, trial periods - there are plenty of options here, but it seems clear this is the most effective way.
I'd argue that you're basically pushing the workload onto the applicant. Now, sometimes it makes a lot of sense to the degree that the applicant can just share a work sample with you--GitHub, published report/paper/other writing, a video of a presentation. In fact, I've hired for non-developer roles where some sort of work sample is pretty much non-negotiable. But recognize this creates biases (and acts as a filter) where something needs to be created from scratch.
One of the most stable and effective findings in I/O psychology is the predictive value of IQ on job performance, which unfortunately is also often legally gray. Maybe companies know what they're doing with coding challenges as an under-the-legal-radar proxy for IQ? Especially if companies are hiring young adults straight out of college with little to differentiate themselves.
The problem with both IQ tests and leetcode interview questions is that in addition to aptitude, they also measure ability to perform under pressure and with a time constraint, skills which rarely come up in the actual job of software development (save needing to fix some bug 10 minutes before a presentation).
Everyone feels pressure differently. For some devs a job interview whiteboard can feel like trying to code with a gun to your head.
I've known brilliant devs who froze under the pressure of an interview. And other brilliant devs who needed to meditate on something, or sleep on it, before coming back with the optimal solution.
I've worked with devs who tended to just run with the first solution that popped into their head, with no patience to mull over a variety of possible approach looking for the simplest one. These devs are fine for straightforward tasks, but become a liability when designing or writing foundational core code.
> I've known brilliant devs who froze under the pressure of an interview. And other brilliant devs who needed to meditate on something, or sleep on it, before coming back with the optimal solution.
Yeah, this is me. Not that I'm brilliant, good side of solid is maybe a better description. But I freeze on whiteboarding and l33tcode. I taught myself binary search and insert sort in high school, but ask me to write insert sort in 40 mins with someone looking over my shoulder and judging and I'll struggle.
Every job I've gotten has come from a take home test.
There's very unlikely to be a general way to interview people that will fit everyone. These kind of cases happen, but how often? 1% of people? 10% of people?
I'm guessing it's far less prevalent than some people seem to think, and that unfortunately, there's no good general system for solving this.
I've had plenty of interviews that didn't involve high-pressure, time-sensitive whiteboards and still managed to draw out my knowledge and ability.
The other possibility, which I don't know why shops don't offer more often, is a contractor position. I've never taken a job that I wouldn't have been happy to work as a contractor for a while first if they asked, and two of my best jobs start out as a contractor.
> I've had plenty of interviews that didn't involve high-pressure, time-sensitive whiteboards and still managed to draw out my knowledge and ability.
I personally dislike "whiteboard" testing as well, and I think it's usually very ineffective. I think some form of work-sample is best, in my 15 years of interviewing we'd usually do: one "open ended architecture" question (which can involve a whiteboard but usually only for diagrams, not code). Then a 1-2 hour programming task, on a computer with full access to the internet.
> [...] I don't know why shops don't offer more often, is a contractor position.
This is usually a completely different position, from many perspectives. It's usually a different budget for the group hiring - they can't easily convert "salaried" positions into "contractor positions".
It's also, in many jurisdictions, a very fraught legal situation. Contractors in general have different rights, but if a company is shown to hire contractors and treat them the same as employees, the contractors can often sue to get the rights of any other employee. This makes sense - it's so companies can't use contracting as a way around whatever legal regulations exist on employment. (Whether those legal regulations are good or not is a different question - but for sure you don't want there to be loopholes around your laws.)
Yeah, General Mental Ability tests are well-supported as the most effective way to measure candidates, but you're right: it's a legal gray area. And even if it weren't, devs absolutely hate them.
So no matter how predictive they are, it's ineffective to use them because you'll identify good engineers... who will never want to work for you ¯\_(ツ)_/¯
I’ve also done 100s of interviews, mostly for mid-senior positions and generally cv-reading and a non-quiz intro interview talking about various technologies and experiences work great.
That said, I disagree about lack of any exercises. I think practical but not too time-consuming take-home assignments are nice and not too annoying (as long as they’re not the first step! That would be wasting candidates’ time.), and I’ve rejected tons of candidates based on the results of those, which is different to the author’s experience.
Overall, I recommend people to really think about who they’re looking for, and heavily adjust their process to that. There’s no silver bullet, and the right process will vary greatly depending both on the seniority and the skillset you’re looking for.
The one thing that's always missing from this conversation is that the "reviewers", "raters", or "interviewers" _themselves_ might not have the skillsets to be able to adequately formulate a roughly accurate opinion of someone's strengths and weaknesses.
Even worse, they might not be able to communicate the actual job requirements correctly, if they even understand the requirements and necessities themselves.
Even worse than that, they often aren't as strong socially as they believe themselves to be. Being able to establish a comfortable atmosphere to connect with someone quickly is rare skillset within the human population, let alone in specific domains.
There is a PRESUMPTION that people in the interviewer seat or management position can and should do this but honestly, after a few decades? I don't see it.
We should be working to solve the "interviewer" problem first, not the other way around.
I interviewed recently at a large GPU-related company, and the interviewer setup a bunch of softball questions.
What's your favorite editor, what language do you like using for scripting, what's your favorite Linux distribution, do you have a homelab...
We were having a decent back and forth and getting along, and then he hit me with the final question, "we really aren't supposed to ask, but what do you like to do in your spare time?"
I mentioned that I had a 2 and 3 year old and joked that I didn't have much free time anymore, and said I liked working in the garage with my hands to get a break from computers.
I didn't realize I gave the wrong answer until an hour or two later (based upon the rest of our conversion and his questions). It would have been the right time to show a few personal projects on GitHub.
Perhaps but I think you shouldn’t feel you were disqualified because you missed the GitHub projects. Stating that you are a father is an honest answer and also sets the expectations on whether you are going to pull all nighters or your dedication matches a fresh grad. As a fellow father I would rather there is an honest match than be surprised half way into onboarding with the stress that ensues.
I try to focus on being the candidate that knows their thing and that will improve the software quality so there are no all nighters. Employers value that and it is something I know I can show up as an advantage. Recommendations mentioning that, help.
IMO the best way to interview a software candidate is just to get them talking about the stuff on their resume. Ask for overviews, diagrams, justifications, etc, etc. If you really know something you will have a very clear mental model of it and be able to discuss it fluently.
Leetcode challenges are a waste of everyone's time. If you are a competent developer then you can google for algorithmic solutions if you really need to, but honestly how often does the need come up, and is googling for suggestions really the hardest part of the project ?!
The only thing that doing well on these type of problems proves is that you've been willing to put the practice time in to get good at them.
Interviewing is the single most divisive problem in software.
I agree with the author on having a plan built for what you're looking for. I don't like the general "culture" questions because they're quite biased ("oh you're reading on HN! Me too!")
I like behavior questions ("tell me about a time when") with a reference scale that explains your. It's not perfect, but it seems slightly more data driven than the rest I've experienced with. I take lots of notes and try to base my verdict on these. I hate leet-code, except at screening level, to validate that the person knows how to type on a computer. I like somewhat job-related exercises for more senior people, code-reviews, and with junior trying to teach them about something new you taught to multiple people and see if they catch it. I betcha half of the people here hate all these methods.
In addition to what you're looking for, I advise trying to see what you miss with people you currently have in your team. (eg if they annoy you, find why and look for what they don't have).
> I recommend that you make a plan for what you want to learn about the candidate, e.g. “are they good at acquiring new skills?” or “do they share the same values as the team?” and then structure the interview around that.
Generally what I want to learn about a candidate is "are they good at understanding plain-English requirements, can talk through their thought process as they problem solve, convert that solution into something that resembles working software, and we can have a decent conversation about what was built." As it turns out, whiteboard-type algorithms and system design questions are very good at this.
Asking if someone keeps up with tech news (and call it a red flag if they don't) is just a way to keep your candidates within a certain social bubble. Which, honestly, is preferable if you're a < 50 person startup. But this doesn't scale. Plenty of software devs at IBM, Amazon, etc are just clocking in and out of their shifts and do a fine job of it. Not everyone needs to make career advancement in the tech industry their life.
Taking a moment to think is totally fine. But it's important to be able to explain what you're going to build before you actually build. Writing up clear design docs and architecture is critical in most senior technical positions.
Obviously the expectation here changes on leveling--with juniors if they just code in silence before explaining their solution at the end, I'm less concerned. But if a senior just starts coding on the whiteboard I'll interrupt them and ask them to explain what their approach is, in English, before coding it up.
I mean... yes? Have you only worked at a company with a single-digit number of engineers? At some point communication is critical and you need to be able to write up a design document before making major changes or additions to code or infrastructure.
> At some point communication is critical and you need to be able to write up a design document before making major changes or additions to code or infrastructure.
... which is typically written after hours of understanding the problem and then minutes to hours of writing out the document. Not 5 minutes after reading a problem statement.
> At some point communication is critical and you need to be able to write up a design document before making major changes or additions to code or infrastructure.
Not sure I've ever had to do that on a whiteboard or under pressure though. Also don't think being good at doing that correlates with writing good documentation
And have less rounds. I don't really understand why companies have multiple interview rounds. One of my friends recently gave three 1 hour rounds all technical.
If you're afraid of making the wrong decision then if one person makes 1/5 bad hiring decisions then 3 people make 3/5 bad hiring decisions not 1/125.
These systems are made to optimize not making bad hires, not necessarily missing out on good hires. Given that, multiple rounds does help. Per your example, if each interviewer has a 1/5 chance of mistakenly approving a bad hire, then three interviewers does in fact reduce that to a 1/125 chance of the company making a bad hire.
I don’t disagree with the sentiment but the tools aren’t the problem, it’s the delivery. A puzzle can might be toxic if it’s a test, but constructive as a gimmick used to explore communication. Likewise for coding exercises.
It’s all pretty easy if respect for the human is genuine, and destructive for all when faked.
But it is a test. "Anti-whiteboarding" people do not seem to recognize that there are many, many candidates who cannot write a loop. Those people cannot be employed as software engineers in most firms, and the wall must be made of brick.
I think you're missing the point of that comment. Yes, you don't want to hire someone to a programming role if they have absolutely no skill or knowledge in the subject and "cannot we write a loop". But that will become quite clear during the interview regardless if you have them doing some leetcode challenge or not. The important part to me is whether they can function in the present team, whether they can employ problem solving skills in a positive way. Sure, let's whiteboard out a problem, show me how you think about it, how you approach it, what steps you take to disassemble it. But also, if you're stuck do you ask for help? Can you discuss the problem without having a solution ready, do you have ideas about how to research it, can you present paths to progress? These are the really important attributes, not "have you memorized this wacky algorithmic solution in the last two weeks". Pure whiteboard tests are just stupid, don't shed light on someone's potential, and carry a context that makes a lot of intelligent high potential individuals freeze up. They select for the wrong traits.
I second this. I've been interviewing candidates for around 6 years and I feel like the number of applicants with very strong communication skills but that cannot perform the most simple programming problems (basic loop, swapping two variables) went up. For my team and most teams at my employer we employ a simple test with a basic programming problem, code review problem and rudimentary system design problem similar to our actual work. We expect the juniors and medior to not succeed at all of these but do want to see how candidates think and communicate if they get stuck, but a junior that can't sort strings in any language of choice is a hard pass. Half of our candidates past initial phone screen fail this test. I do wonder though if this has something to do with the local (NL) market.
Each company I’ve worked with has attracted a slightly different group of applicants, so I find it easy to believe NL would be different yet.
Whenever I start thinking the problem is the candidates I start asking myself “why?”. Why we came to the conclusion we did, why they applied, why we need more people on the team, etc.
It’s not your local market and this is not new. I interviewed folks at google head office in 2012 so we had people from all over the world and different experience levels and it was the same thing
You are so wrong about that. I’ve had people recite gang of four design patterns and talk about template metaprogramming then absolutely fail at writing a doubly nested loop in their language of choice. Coding is a skill that you can only really master with practice not just reading books and blogposts (although that definitely helps)
I asked very basic coding questions that didn’t require “advanced” algorithmic knowledge like DP which are popular at some shops
Exactly. At the other end of the spectrum, having a conversation also exposes the “over prepared”. There are some (few) folks that practice leetcode all day, but that doesn’t translate to team fit and on the job value creation. Not being able to communicate while working (not continuously, small amounts with good effect is fine) is a bit of an orange flag for me.
I realize now I just won’t be able to understand the crowd that wants an easy ride through interview loops and refuse to be put in a hot seat. Do they not care at all who they end up working with? Because you will 100% end up having half of your team incapable of writing software in such places
I don’t accept the premise that being kind and respectful in an interview leads to underperforming teams. My current team is way, way into the “exceptional” range by common measures (DORA, etc). I don’t characterize what I do as “tests” because there is no wrong answer. I’ve never worked on software that didn’t have competition, and so literally different “right” answers.
Maybe it is a test as there is an evaluation, an assessment of fit and ability of compliment the team’s current skills and way of working. That evaluation is uncompromising, not an “easy pass” by any means.
“Kind” as in passing people who probably wont cut it? You’re only being kind in a feel good about yourself sense and just setting them up for getting pip’ed later on. What’s so kind about that?
That is not what I wrote, and not at all what I meant. A high-pressure "test" is cruel to folks who simply don't test well. Simple as that. Interviews can get at the skills of someone without putting them through that.
I can't, so far, think of anyone who faked it past my interview teams over the years. It's not that hard to figure out if someone can actually code, has a sincere interest in it, respects others, craves improvement and measures their work by the value created. I really don't care if they test well.
And, like I said, the bar is actually very high to join my teams. But my teams aren't prototype or manifestations of any theory of teams -- it's just folks getting the work done and enjoying it along the way.
A team of different folks will need different things by definition and still be valuable individuals, each different, each worthy of respect.
My interview questions eventually settled to having the candidate bring in a sample of their code and then teaching me how it worked.
We'd talk about what it did, what they thought they'd change to make it better, and so on.
Learned a ton really quickly and didn't waste anyone's time.
The last tech job I interviewed for they had me spend a lot of time working a sample application. When I thought about how many person hours were going into that across all the candidates, I immediately felt the company didn't have a sense of efficiency. Sure, it didn't cost them, but the fact they were fine just wasting time like that was telling.
And it goes all the way up to the top. I can name CEOs who were great hires who not at all an obvious fit for a tech company and I can name CEOs who certainly looked good on paper who were disasters.
Whiteboard tests are not a good way to evaluate people. It doesn’t tell you if they’re going to be a leader. It doesn’t tell you if they’re going to take an initiative. It doesn’t tell you if they’re going to be a person that complains all the time. It doesn’t tell you if they’re going to be able to solve problems they haven’t seen before which by nature will be all problems they encounter.
Whiteboarding interviews are a good way to make the applicant feel bad about themselves so you can lowball them in salary, however
I agree, but the same can be said of any other interview technique. There's no rock solid, reliable way to determine in 30-60 minutes whether a candidate is going to be a good hire. I'm not in favor of whiteboards, but I'm also not necessarily in favor of anything else.
I'm told that in the early days of the German branch of takeaway.com they would conduct interviews late and invite the interviewee to their after work beer and only hire the person if they thought they would enjoy future beers with them. Obviously only useful to somewhat determine personality but in my experience that is often a better predictor for a good hire than tech skills.
I think the biggest issue with this is that you leave yourself open to bias. The person you're most likely to enjoy food/drinks with is the person you're most likely to be friends with. You'll end up with teams hiring via popularity, and possibly excluding those from other cultures or who have different personality traits.
That is an unprofessional, abusive way to treat job applicants. It's going to filter out those who don't drink or who have evening family obligations. Any hiring manager who engages in these shenanigans should be ashamed of themselves. And in the USA at least, they could potentially be exposing the company to legal jeopardy.
You don't need leetcode challenges to weed out people who are lying or totally incompetent. I simple whiteboard test such as reversing a list will do the trick!
>>> Whiteboard tests are not a good way to evaluate people
>> How do you make sure you don't hire these people?
> I simple whiteboard test such as reversing a list will do the trick!
Agreed. I did a lot of hiring interviews of experienced embedded C programmers. A few of the applicants could not read two bytes from memory and combine them into a 16-bit integer, no matter how much handholding they got.
But they had good resumes and interviewed well other than that.
So some kind of check of their actual technical skills is valuable.
We spent a much larger portion of the interview having them explain things like "how would you implement a firmware upgrade architecture where the power can be lost at any time during the upgrade?"
Why are you hiring the guy if the job is that easy? Just because someone can do leetcode questions doesn’t mean that they can design a system or even design an API. That actually requires creativity not memorization.
I wouldn't recommend hiring someone (except perhaps for an entry level job) based on ANY kind of whiteboard test. For a senior person it's their experience that counts, which is why the interview should be around what's on their resume.
The list reverse (or anything similar - count word occurrences in a list) doesn't qualify someone for the job, but is a good way to quickly tell if someone is lying about their experience, which unfortunately some do, especially for low paying contractor jobs. For stuff this simple I'd expect people to just write it down in 30 sec once they've asked any clarifying questions (such as can I use list.reverse() - no).
I completely agree with you, but it seems like even for senior or staff software positions they still whiteboard interview you.
Here’s the problem though: most software development is done alone and most software engineers are introverts. They do not like to be put on the spot and be the center of attention. When you ask them to do things under pressure (which basically means your company is always having fires to put out - management problem) they can’t think straight because half their mind is focused on how you’re judging them because the interview matters to them much more than it matters to you. You’re not giving them a fair test.
I don't think it's fair to give any kind of challenging test in an interview, but something really simple like this is different - they are not really being asked to solve anything, just to show that they can indeed use the language they've got listed on their resume.
And if you're hiring for the latter, don't give the candidate the impression you're looking for the former. It's very insulting to think you're being hired to take initiative and provide leadership for the team to solve problems and then be told you aren't clicking on the right Jira buttons to make the metrics look good.
At some point the candidate is going to need to write some code. If you haven't been involved in hiring, you might not know how little proficiency it's possible to have in actually writing code, while having a beautiful looking resume.
Clicking buttons in jira alone isn't going to go very far.
I've been on both sides of interviewing, so much so that I can almost tell just be looking at subtle things on a resume or in interviewing whether or the person has real programming skill. In my early days, I hired some real stinkers, and learned from those errors. I've worked with people who wrote code that made The Daily WTF[1] (I know, because I submitted it) I know how to reject candidates that check the boxes that HR screens but can't code. I've consulted (mercifully briefly) at organizations where programmers whose primary skillset was using copy/paste but were able to jump through all the Jira (technically Azure DevOps: same shit, different name) hoops were given ownership of key parts of a system.
You might find it interesting to track what happens to the people who do your coding exercises.
For the ones you hire, how do they do in the company?
If you can see happens to those you reject, via whatever online profile they have, you'll get a good sense of when you missed a good hire.
My strategy was always the same, ask technical questions and go deeper with each question until the applicants says something like "I don't know but I would do xyz to find out."
But I always warn first that there's no problem if they don't know the answer so they know first-hand they can say I don't know.
Very easy to detect bullshitter that will invent stuff or those on the first level of dunningkruger.
And the best moment of my life was when I interviewed someone who had all the answers and the explanation of why it was working this way.
I told hr that we had to hire him in a higher position than me!
I like the strategy, and I do something similar. You really just need to ask them about their experience and listen to them talk. The people that know what they’re doing will gladly go into too much detail whereas the impostors will say a lot of jargon, but not explain how things work
It took like 2-3 minutes for mine to load. Here's the text:
-------------------------
> Anders Borsch (@anders@mastodon.cyborch.com)
I have interviewed 100s of candidates for software engineering positions.
I’ve done take-home tests, in person challenges, pair programming with the candidates.
All of them were awful experiences for me and especially for the candidate.
I can only think of a single instance where a code challenge exposed a poor software engineer and I could definitely have made the same assessment just by talking to them.
Lately I’ve stopped doing any software or mental puzzles.
I don’t do any of that when I interview designers or QA people or HR people, so why would I be particularly toxic towards software engineers during the hiring process?
Instead, I actually read their resumes (which is significantly quicker than doing interviews, asking them to repeat the same information), and then I ask them questions like:
- Where do you get your tech news?
- How do you learn about new technologies?
- What do you most appreciate in your coworkers today?
- What is a perfect workday like for you?
I specifically avoid trap-style questions like “what is your greatest weakness?” or “why are you leaving your current job?”
I recommend that you make a plan for what you want to learn about the candidate, e.g. “are they good at acquiring new skills?” or “do they share the same values as the team?” and then structure the interview around that.
Be a non-toxic manager. Make your company look good during the interview process. Get better candidates.
My first 3 or 4 tech jobs started with interviews like this, which seemed to work fine. Then everyone fell in love with coding questions. It's funny to think some hiring managers might be going back to interviewing like every other job.
Mastodon, like many other db backed web apps (notoriously WordPress), is a very brittle application when you throw a lot of unexpected load at it and when it's run on a single server.
I've never used Mastodon before, but this was a terrible first introduction. Does that mean that different Mastodon instances will just go up and down depending on how their server is doing that particular moment? There's no caching or CDN?
Modern Wordpress hosts do both, and it's usually very very quick to load.
Many people run theirs on a small VPS, raspberry pi, etc. and there are a number of 'paid hosting' providers for Mastodon servers too.
In practice, this all means yes: unlike a blog, it is a complex web application, and caching and whether a CDN is in use are up to the person hosting it.
Mastodon can also be fairly expensive to run, depending on how many images/videos people upload, traffic they bring to the instance, etc.
Oh wow, I didn't realize that was part of its philosophy. I thought it was more a Usenet-style system, where small/medium companies (like ISPs) would host instances. But individual users doing it? That sounds like a surefire recipe for bottlenecks and unreliability...
Well, yes. I suppose one could enable caching in Nginx or some other reverse proxy but Mastodon itself is just the app, there'd be no point for it to implement caching or a CDN when the admin can hook that on in front.
But basically, it's an essential part of a modern Wordpress stack (lol, what an oxymoron, I know), not an afterthought. It's not something a lone-wolf admin using a generic Varnish/Nginx config should do, but part of a large a community-driven effort to improve Wordpress performance. These plugins are very well integrated into the Wordpress base system and cache many parts of it.
This isn't so much to defend Wordpress (whose ancient architecture requires the use of these techniques), but to express surprise that a modern software like Mastodon doesn't have this built-in.
Sorry, this definitely wasn't meant as a complaint against your post! I was just surprised at the Mastodon architecture, having never seen a post in the wild before. I didn't know they were usually self-hosted.
Some first level of the hiring funnel (HR, recruiter, or something) screens the candidate by looking at the resume and maybe doing an initial phone screen, and then this person gets dumped on the interviewer, and they 100% could be anywhere on that skill scale. So you always seem to have to start at "how do you turn a computer on?" and "write code that finds a substring in a string." It's really frustrating for the interviewers and I'm sure also insulting to actually good candidates.
I've never been in one of these businesses, but I bet if a hospital interviews a doctor, they can at least count on basic knowledge of anatomy, and if a law firm interviews a lawyer, they can assume the person has read a law book. We have no such guarantees in software.