With a homework assignment that is not timed, you let the candidate do her best possible work in her ideal environment, and yes, while a work environment is sometimes chaotic, when she needs to get her work done, she is going to make her environment as perfect as possible.
I'd like to second this comment as someone who was recently laid off, did a bunch of interviews[0] and landed a job I'm enjoying very much. The company I picked had a "homework assignment". It wasn't a real project that had any useful purpose to them, which I've seen suggestions about doing[1], but a simple web based app.
At first, I was a little less enthusiastic about the approach -- I have GitHub repos and other examples of competency out there -- but as I thought about it, this "test app" was in line with what they were looking for and when they described the approach, I liked it. The first thing they did correctly was provide a basic spec sheet. It wasn't overly specific and I think this was by design. It was basically a CRUD web app, with a statement of "feel free to get creative; do as little or as much as you feel it needs". I provided a self-hosted CRUD app that worked on multiple devices, syncing state between them in real-time, documentation, an installer and everything else that was needed to make it a complete product. Most of that wasn't on the spec sheet, but in line with thinking about the project as "something I'd deliver", those were things I'd consider to make it complete and it also gave me an opportunity to demonstrate competency in different frameworks, provide examples of functional/OOP design and other areas that gave them an adequate look at what I could do. The app, itself, was a really minimal web app that took me a couple of hours to write, but after all of the additions, it landed in at a day and a half or so -- not a terrible amount of work for someone out of work and worth it since I landed the job.
Frankly, given the choice between shoulder-surf coding, whiteboard coding[2] and test projects, I'll take the test project. I hate whiteboard interviews and thankfully there were none of those this time around. I've had two whiteboard interviews in the last five years. In one case, I got lucky in that the first algorithm was one I had written into an app I had on a private (personal) BitBucket repo, so I pivoted with "I can marker up your board here or I can show you a good implementation that I did a few weeks ago". We spent the rest of the interview with them asking questions about my choices and that was one of the most enjoyable interviews I've had. I didn't take that job. The other, I didn't get as lucky and actually ended up turning down the job on my way out of the interview room[2].
I agree with the author about having empathy for the job seeker. This empathy, though, works both ways. In my case, I had filled in for a dev team in a pinch -- at the last minute -- to run an interview. They hired the guy on my recommendation and he ended up being a great employee so I was pulled in to most of the interviews for several of the dev teams from that point forward. Never mind that I did development on an ops/security team and didn't work with any of these teams, making whether or not they hired a particular person -- or anyone at all -- of now benefit to anything I was responsible for. As a guy who had an already overbooked week, I didn't enjoy being pulled into these and I really didn't appreciate the massive number of woefully under-qualified candidates or the fact that all of that time was often spent on a great candidate who turned down the job in the end. Because of this, I express thanks at various times during the interview process and specifically mention how grateful I am for the time taken out of their already busy work-life. I was glad for the opportunity to interview at all of the places that chose to interview me so I simply made that known. The cool thing about that is later interviews felt more at-ease with the team as a result[3].
[0] I had quite better luck than the author, but did fewer interviews, and was seeking a position that was right in line with my experience.
[1] I like this idea, too, provided compensation is provided -- after all, if you intend to use the product of the work a candidate performs, they're technically a contractor and I wouldn't do work like this for free.
[2] I actually do pretty well on my toes -- I'm good at rationalizing out loud, and breaking up the tension with a little humor (to the point that I've been given hints at answers when I was stuck). Despite turning down this job, they made a really good offer to me two weeks later that I declined -- the job itself was in a domain I wasn't excited about, but the whiteboarding didn't help.
[3] Felt being the operative word -- these were later interviews which means the team was already more interested in me as a candidate than in the first interview. I also remember the first time a candidate specifically thanked me for the time I spent interviewing him and very likely projected that onto my interviewer. But at the same time, it increased my confidence so if it was all in my head, I'm OK with that. I also feel that it was "the right thing to do".
At first, I was a little less enthusiastic about the approach -- I have GitHub repos and other examples of competency out there -- but as I thought about it, this "test app" was in line with what they were looking for and when they described the approach, I liked it. The first thing they did correctly was provide a basic spec sheet. It wasn't overly specific and I think this was by design. It was basically a CRUD web app, with a statement of "feel free to get creative; do as little or as much as you feel it needs". I provided a self-hosted CRUD app that worked on multiple devices, syncing state between them in real-time, documentation, an installer and everything else that was needed to make it a complete product. Most of that wasn't on the spec sheet, but in line with thinking about the project as "something I'd deliver", those were things I'd consider to make it complete and it also gave me an opportunity to demonstrate competency in different frameworks, provide examples of functional/OOP design and other areas that gave them an adequate look at what I could do. The app, itself, was a really minimal web app that took me a couple of hours to write, but after all of the additions, it landed in at a day and a half or so -- not a terrible amount of work for someone out of work and worth it since I landed the job.
Frankly, given the choice between shoulder-surf coding, whiteboard coding[2] and test projects, I'll take the test project. I hate whiteboard interviews and thankfully there were none of those this time around. I've had two whiteboard interviews in the last five years. In one case, I got lucky in that the first algorithm was one I had written into an app I had on a private (personal) BitBucket repo, so I pivoted with "I can marker up your board here or I can show you a good implementation that I did a few weeks ago". We spent the rest of the interview with them asking questions about my choices and that was one of the most enjoyable interviews I've had. I didn't take that job. The other, I didn't get as lucky and actually ended up turning down the job on my way out of the interview room[2].
I agree with the author about having empathy for the job seeker. This empathy, though, works both ways. In my case, I had filled in for a dev team in a pinch -- at the last minute -- to run an interview. They hired the guy on my recommendation and he ended up being a great employee so I was pulled in to most of the interviews for several of the dev teams from that point forward. Never mind that I did development on an ops/security team and didn't work with any of these teams, making whether or not they hired a particular person -- or anyone at all -- of now benefit to anything I was responsible for. As a guy who had an already overbooked week, I didn't enjoy being pulled into these and I really didn't appreciate the massive number of woefully under-qualified candidates or the fact that all of that time was often spent on a great candidate who turned down the job in the end. Because of this, I express thanks at various times during the interview process and specifically mention how grateful I am for the time taken out of their already busy work-life. I was glad for the opportunity to interview at all of the places that chose to interview me so I simply made that known. The cool thing about that is later interviews felt more at-ease with the team as a result[3].
[0] I had quite better luck than the author, but did fewer interviews, and was seeking a position that was right in line with my experience.
[1] I like this idea, too, provided compensation is provided -- after all, if you intend to use the product of the work a candidate performs, they're technically a contractor and I wouldn't do work like this for free.
[2] I actually do pretty well on my toes -- I'm good at rationalizing out loud, and breaking up the tension with a little humor (to the point that I've been given hints at answers when I was stuck). Despite turning down this job, they made a really good offer to me two weeks later that I declined -- the job itself was in a domain I wasn't excited about, but the whiteboarding didn't help.
[3] Felt being the operative word -- these were later interviews which means the team was already more interested in me as a candidate than in the first interview. I also remember the first time a candidate specifically thanked me for the time I spent interviewing him and very likely projected that onto my interviewer. But at the same time, it increased my confidence so if it was all in my head, I'm OK with that. I also feel that it was "the right thing to do".