But easily revised to "Do you want to come with us?"

Not that I particularly think the rules are inviolable. I see them as guidelines: if you're using the passive voice then make sure that it won't be better in active. If you're ending with a preposition, then perhaps there's a clearer way to phrase it.