I live at sea level, and I've been up to 10,000 feet five times. I've felt fine for four of them, but the fifth time, some light physical activity (~60 minutes of walking) put me flat on my ass, made my feet feel like lead, and made me want to do nothing other than to lie down and sleep.
Different people react to altitude differently.