LLMs don't understand security 101, or anything else for that matter. It shouldn't be surprising if they do something like this.