> although personally I've always wondered why this is desirable

Carry-on space. I have yet to see a truly full flight where at least some carry ons aren't gate-checked.