To call not being accepting of any incarnation of racial violence "zero-tolerance" has a really unfortunate air of dismissal to it. I, for one, am not a middle school hall monitor. There are things that a (relatively) advanced, civil society like ours should not accept. Just because you say something and may or may not mean what someone else meant does not remove the broader social context. The word is racist because of the social context and when you use it you evoke that context whether you understand it, agree with it, or not.
Edit: to be clear, my point is that evoking such an idea or meaning is violent towards others and an individual doesn't get to decide that this isn't the case. Believing you can is a hallmark of privilege.
Given that racism is a universal thing and often happens intra-minority (African-Americans and Jews are an (in)famous example), your statement would pretty much mean that just about every demographic exhibits a hallmark of privilege by sheer virtue of ignoring broader social contexts in a racial slur.
Edit: to be clear, my point is that evoking such an idea or meaning is violent towards others and an individual doesn't get to decide that this isn't the case. Believing you can is a hallmark of privilege.