Hassle? Stigma? Hardcore drug addicts will do anything to get their fix. I have known heroin addicts, for example (one was a friend who sadly got sucked into that world). When you're zonked out on heroin most of the time, the way society perceives you is of very, very little consequence.
As another commenter mentioned, I'd imagine a lot of drug trade (maybe not in heroin, but definitely in cocaine) is not hardcore drug addicts, but occasional (or even regular) users who live (relatively) normal lives.
