I'm with you. I think you should have to pay extra for containers and utensils. Make it a deposit and suddenly single-use plastic is multi-use.

Or, as you say, let people provide their own container. In any case, if you charge people for the externality you will likely end up with less of it.