I was once told my Dodge was a "drug dealer" car. I laughed, saying any sane drug dealer would choose a boring, nondescript, dime-a-dozen car that did not stand out. My Dodge attracts a lot of attention.
Sounds like there's a difference between drug dealers that do and don't know how to keep under the radar. And the person who said that to you, only knows of the unwise ones? :)