Taxi, takeoff and landing are the most critical parts of a flight. If something goes wrong and you can't hear instructions, chaos etc because your distracted with your music then you are a danger to yourself and your seatmates around you.
Is that actually an official position though? I've only ever heard it as post hoc rationalization. There might be a bit of sense to it, but on the other hand, it's not hard to jostle someone into paying attention, as I'm sure your neighbors would. They don't prevent you from going to sleep before takeoff or landing, and that's at least as difficult a context switch.
Nor do they forbid you from being deaf during takeoff. They may ask you to change seats, so perhaps it is reasonable to ask that people sitting in those particular seats not have headphones on, but otherwise listening to music seems fine to me.
At least in Canada, you're only allowed to have headphones on if they're earbuds (easy to quickly remove) and connected to the in-flight sound system (so you still hear announcements). I haven't paid attention to the onboard safety dance in ages, so I don't recall what it is elsewhere.
