Hey, you guys check out that new restaurant down the street? Is it any good?

That depends on the person. I like when other people come to my desk and ask me for help on a whim, even when I'm in the middle of something. It doesn't destroy my productivity, and often it even gives me ideas how to make my work more useful. I do respect when other people don't want to be interrupted, though.

If you stop what you are doing then, by definition, it is breaking your workflow and stopping your productivity.

I usually take it as an opportunity to get some suggestions on whatever I was working on when they stopped me.