It works well in my experience, but we have only used it at smaller scale. 2-3 days working together with somebody is already way more telling than stupid interviews, buzzword compliant skillsets and unrealistic tests.
How would it work in "your experience" if you were the job seeker and not the interviewer? Taking a week of PTO to interview and then not get the job is a pretty big sunk cost. I guess you can only do one of these a year.
If you make weekends and evenings available, it lessens the cost of the job seeker. If you allow for remote work using one of the many online collaboration tools, it lessens the cost for both the job seeker and the employer.