Part of it, historically, is the "reproductive" part. Cooking food for you doesn't result in a pregnancy and a kid who then grows up with a mom who was by definition in the position to consider sex work the best choice. And in fact if you're the child of an exploited sex worker (as opposed to the independent entrepreneur), you are a child in high danger of being exploited yourself, because exploiting you can be quite lucrative.
Now, we could help children by helping people and providing social services -- but it's easier to just make prostitution illegal.
What if I pay a security guard (or a soldier for the army for that matter) with an implicit non-trivial risk of being killed by an intruder he's there to repel? That seems like an even more significant "act" that will also affect their children, as well as a lot of others potentially. Not speaking of the gravity of the act itself for the person being paid. Should that also be illegal?
Now, we could help children by helping people and providing social services -- but it's easier to just make prostitution illegal.