Even if you had downvoted me, by the simple fact that you provided some explanation (however minimal) is loads more engaging and interesting than providing an opinion, and coming back to see it scored at -3 or whatever.

I would understand the vote, and would still feel interested in engaging. I would view the downvote as a punctuation or an emoticon on your comment. Not a ricochet bullet in my living room.