How want you add a delete button with out adding a complete login system? Or do you want to allow everybody to delete every comment? And of course this is also doable, but the question is, is it worth for a non profit (non tracking) blog? Probably not. Is it worth for Facebook and Google? Sure.
Exactly. I'm not arguing it's impossible, but that it imposes a meaningful additional burden on small operators without any real benefit to privacy.
Personally, I don't even think people should have the right to go back and delete a comment from years ago, which might have started a whole interesting discussion. But the EU requires that I think through such a system, including finding a way to identify them as the commenter and purge their PII from all logs/backups/caches as well.
