The problem is that often middle-aged cannot deal with that anymore; an unlucky injury on/after that age can ruin the rest of your life and shorten it considerably. So many people who did things like ski/snowboard all their life but fell wrong and messed up their knee over 45 and now they cannot really do most sports they enjoyed anymore. People are not that into change usually when they get older so it's often quite physical but also mental issue for them.