I'm thinking if the org that trained the model, and is doing interesting research of trying to understand how LLMs actually work on the inside [1], their caution might be warranted.

[1] https://www.anthropic.com/research/tracing-thoughts-language...