This has been a security vulnerability since day 1 with these models, yet collectively the people who use them just simply don't seem to care about the security implications. Its especially problematic given that people let AI agents have full unrestricted access to their system
Its going to take even more data breaches for the AI crowd to finally care, but to a large degree I have absolutely no sympathy. You know what you're signing up for if you sell yourself to anthropic or openai
I think it's all based on ease. I also think that, eventually, everyone will just move to containerized AI, with a dropbox like system to get things in and out, and some fairly minimal network access. This is close to how 99% of people use AI already, through apps like the ChatGPT app.
Its funny because I have been asked by some folks at work as to why I dont just 'allow all commands' and instead review each one, and then I let my colleague in infosec know who asks that so they are aware of dangerous users.
Because it's extremely annoying and full of false positives. The vast majority of the commands are fine, analyzing each and every command the AI generates will quickly give any human alarm fatigue.
It's more ergonomic to put them in a fully isolated virtual machine and let them cook.
The user wants to reformat his hard drive. I can use the bash tool for this. Wait, does the user want a full reformat or just deletion of all files? Ah, the user only wants to delete his home directory. I can do this with the command "rm -rf $HOME" using the bash tool.
> people let AI agents have full unrestricted access to their system
Not everyone is like this. I've never ran Claude Code unsandboxed. I took the time to firejail it since day one. One of the first things I did was ask it to write a virtual machine orchestration tool so I could virtualize entire projects and their associated AI sessions. Mitigates supply chain attacks too.
I managed to rig up a firewall and it withstood a Fable code review. I'm now writing a custom network stack for it in Rust so I can firewall every single network connection.
Nobody cares. Nobody with the power to do anything about itcares because they don't bare the costs.
You can see this pre AI with how companies handle data breaches. They never cared about security outside of reducing financial liability. Why would they let such petty concerns, which they are already insulated to the hilt against with all the well practiced liability reducing procedures and checklists, prevent them from capitalising on "massive productivity gains"?
Its going to take even more data breaches for the AI crowd to finally care, but to a large degree I have absolutely no sympathy. You know what you're signing up for if you sell yourself to anthropic or openai