Last year, a company that was interested in hiring me as a software developer asked me to build a food delivery application for a fictional restaurant, as a way to test my coding abilities. I was a bit shocked. The time commitment for building an entire application from scratch can be substantial, and the homework assignment didn’t pay.
After a long weekend of work, I was so exhausted and miffed that I gave up. I told the interviewer I wasn’t interested in the job, but the reality is that I was dismayed at the interview process.
I queried my network, asking if anyone else had been given homework assignments. It turns out I wasn’t alone. Developers all over the U.S. had encountered the practice, spending anywhere from a few hours to over three days working on their unpaid interview assignments. In a talk given at PyCon last year, software developer Susan Tan said she spent as long as 32 hours on homework, only to be rejected because it was missing a feature not even stated in the original requirements. She says she usually didn’t receive any feedback at all, a common gripe among the developers I queried. The most extreme example I heard about directly came from Erik Umenhofer, who lives in Silicon Valley and told me that he’s encountered tests that required a solid three days of work.
“In the Bay Area people just get used to it,” he said. “When one company sees another company does a hiring practice, they all start to adopt it.”
For my first full-time development job back in 2010, I endured a 6-hour-long unstructured interview. As was typical for job interviews in the tech industry at the time, I chatted with every member of the team about my background and skills. Some of them had prepared questions, others hadn’t.
Numerous studies now show interviews like these are a potential source of bias and do not accurately evaluate candidates.
Later on in my career, in part to avoid this potential bias, the tech industry introduced other forms of testing with the goal of testing skills, such as “whiteboarding,” which involves asking job candidates to jot out the solutions to brainteaser questions such as finding the first 25 prime numbers of a sequence.
There was backlash against that, too, as critics pointed out it hardly resembled how most programmers work on the job. Then Google $GOOGL, the most famous proponent of these tests, delivered “whiteboarding” a final blow when it admitted its own studies showed the puzzles weren’t even very useful in evaluating skill.
“Take home tests” emerged in this vacuum as an opportunity for developers to use their own computers to work on solutions to relevant problems in the comfortable environment of their home.
Completing days of free work as a requirement for applying to a job is a burden for anyone, but it may also deepen biases against already underrepresented groups in tech, such as women. Women still perform most child care in this country, leaving them with much less free time to do these tests. As software developer Gabriela Voicu told me, many women don’t have time for “code challenges that may or may not go anywhere.” She was especially irked that some companies claim these challenges remove bias and create a fairer process. “If that is their goal, that is an imperfect solution,” she said.
Because homework is so new, there are no studies comparing them to other methods of hiring. But some hiring managers already contend that there are ways to evaluate programmers that are less time consuming.
Pete Holiday, an Engineering Manager at CallRail in Atlanta, used to use homework as part of job interviews before realizing that he was ruling out good candidates. Some told him they didn’t have time for homework. Others may have never gotten to that point. “It’s way more inclusive to just have someone come to the office and talk to them,” Holiday said. “You’re not counting on them having time, or a computer at home. We have candidates with sick family member, single parents. Without the homework we can cast a wider net.”
Holiday also noticed that some candidates weren’t familiar with the type of framework the test expected them to use, but they were more than capable of learning it.
Holiday’s company now uses a structured “live-coding interview” format that involves reviewing code together and talking about it. The process typically takes between 30 minutes to an hour.
Alistair Davidson, a Technical Architect in London, says his company made the same switch after learning that a “small” homework assignment they’d developed took a candidate 16 hours to finish. “[A live-coding interview] gives you a better feel for what it’s like to work directly with someone and try to solve a problem with them” he says.
In the meantime, many developers, especially at more senior levels, are starting to simply say “no thanks” when handed homework assignments. And that’s the approach I will also take when I’m on the job market again.
