I don't understand how take home interviews are being held up as a better solution. They have two major flaws:
They lower the cost for the company to zero. So they can widen their pool, and take a chance on more candidates, with no extra cost incurred for them. If you take this to the extreme – every company asking every candidate to complete a take home – then the result is far worse for candidates.
There is no feedback mechanism for candidates during the process. I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable. On occasions where we had a system for capturing progress, I found that most candidates far exceeded the time recommended to them. Some were spending up to six hours. The frustrating thing was the extra time never made a difference to the outcome. I could tell from their initial hour whether they were going to pass.
As a hiring manager I've used them, but work hard to be considerate to the candidate. I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.
My company does take-home interviews (or rather, we give candidates the option to do them; nearly all candidates choose take-home instead of a live coding session) and I don't think I agree with the "lowers the cost to zero" point. First of all, they are never the starting point; we only give a take-home problem to a candidate after we've had an initial discussion with them.
When we get a take-home submission, we essentially run it through our code review process, and as one of the engineers who does reviews, it usually takes me nearly as long to thoroughly examine a homework solution and write up my feedback for internal discussion as it would to do an in-person interview.
Being able to review a candidate's code on my own schedule rather than being yanked from the middle of some knotty problem to go do an interview at a fixed time is a big quality-of-life win for me. But the rate limit isn't much different than it would be with regular interviews.
All that said, it's totally possible our process is super atypical.
At Caltech, most exams were take home with a time limit of 2 hours. Some were "infinite time". The students hated the latter, because it means there was no end to the time you needed to spend on the exam, meanwhile there were other exams, etc., to take.
With a 2 hour exam, there's an end to it, and you know how much time it should take to do the problems. Much better.
It's even worse if you're being graded on a curve, as you would be in a job interview. You're competing against others who will spend infinite time on it.
Like you said, "there were other exams, etc., to take.". Presumably other competitors have other exams as well? You'll thus be competing against people with different time management skills.
> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable
> I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.
Why don't you just let candidates decide for themselves how much time they want to spend on the task? You can set the expectations, saying that you expect that the task of a given complexity will to take between 1 and 2 hours; but candidates should be free to spend as little or as much time on a task as they like. Some candidates will value their time and spend no more than is advised; others will be sufficiently obsessed by the task to spend more time. Why should the latter group feel guilty about exceeding the recommended time limit?
You're trying to evaluate a candidate's ability to accomplish something with finite resources. If you allow someone to put additional time into it, you're biasing the results towards the more desperate and less in-demand individual.
- If the more desperate and less in-demand individual writes bad code, or makes poor judgements, you will be able to see it in their submission, won't you?
- Whereas if such an individual, because of their desperation or lack of demand, has enough time and motivation to elegantly solve the problem, then what's not to like?
I think a greater concern would be that with take-home assignments it's impossible to be sure you are actually evaluating the right individual. How can you be sure they didn't receive help from someone else?
The programming test is just one aspect of a candidate, in my estimate about a third of the total "score". Take-home exams bias towards evaluating programming ability but bias against other desirable candidate qualities such as experience.
Their desperation to get the job doesn't necessarily carry over to their day-to-day programming. If they take a month to do a week-long task, I would want to know.
In terms of cheating, that can be easily resolved by asking detailed questions about the candidate's submission. It is also helpful to show that you took the time to review it. (There's nothing as frustrating as taking a day for a take home test and getting no response).
> but bias against other desirable candidate qualities such as experience
Won't you see, in the code submission, at least some evidence of experience? The way the candidate organises their code; the overall style; the third-party libraries they choose to use (or not to use); the tests they write (or don't write) — doesn't it all, combined, speak of experience or of lack thereof?
There are still other stages of the interview, where you can assess candidate's experience more directly, by asking about what they worked on.
I agree that it's useful to know when a task that should have taken x long takes 5x or 10x; but it may be because the candidate set a higher bar for themselves.
Totally agree. I've almost never completed a take home project (even though I liked the concept) because there were always other companies I liked that didn't demand as much of my time whose interview process moved faster and who gave me offers before I finished the take home interview process.
> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable
I'm going totally nuts about this particular problem. I have written in as clear terms as possible that I really don't want the candidate to spend more than 2 hours, but I don't feel I am going to convince anybody - especially as this is a (mini)game programming test, and with games you can always add one or ten more things.
They lower the cost for the company to zero. So they can widen their pool, and take a chance on more candidates, with no extra cost incurred for them. If you take this to the extreme – every company asking every candidate to complete a take home – then the result is far worse for candidates.
There is no feedback mechanism for candidates during the process. I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable. On occasions where we had a system for capturing progress, I found that most candidates far exceeded the time recommended to them. Some were spending up to six hours. The frustrating thing was the extra time never made a difference to the outcome. I could tell from their initial hour whether they were going to pass.
As a hiring manager I've used them, but work hard to be considerate to the candidate. I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time.