Yeah, special was already an insult when I was in school in the 90s. Didn't realize challenged had become one already. Have they made a noun out of it or just an adjective? Retarded produced both, which is one probably helped it spread.
And that's one of the problems with PC. In that regards, it doesn't work. If we call "orange thinkers" those which IQ is around or above average and "green thinkers" to the other group, then sooner you'll hear "What? Are you a green thinker?"