Every 6 months since chatgpt launched, everyone keeps telling me that LLMs are going to be amazing in a year from now and they'll replace programmers, just you wait
They're getting better, but a lot of the improvement was driven by increases in the training data. These models have now consumed literally all available information on the planet - where do they go from here?
The "time to amazingness" is falling quickly, though. It used to be "just a few years" a few years ago, and has been steady around 6 months for the last year or so.
I'm waiting for the day when every comment session on the internet will be full of people predicting AGI tomorrow.
As far as I understand, coding ability of AIs is now driven mostly entirely by RL, as well synthetic data generated by inference time compute combined with code execution tool use.
Coding is arguably the single thing least affected by a shortage of training data.
We're still in the very early steps of this new cycle of AI coding advancements.
Yeah... There are improvements to be made by increasing the context window and having agents reference documentation more. Half the issues I see are with agents just doing their own thing instead of following established best practices they could/should be referencing in a codebase or looking up documentation.
Which, believe it or not, is the same issue I see in my own code.
Give the LLM access to a VM with a compiler and have it generate code for it to train on. They're great at next.js but not as good with swift. So have it generate a million swift programs, along with tests to verify they actually work, and add that to the private training data set.
As you can see from the other commenters on here, any perceived limitation is no longer the fault of the LLM. So where we go from here is gaslighting. Never mind that the LLM should be good at refactoring, you need to keep doing that for it until it works you see. Or the classic you’re prompting it wrong, etc.
Let's hope you are right, and that the limitations of AI will never improve, and that all of us get to live out our full natural lifespans without AI xrisk
They're getting better, but a lot of the improvement was driven by increases in the training data. These models have now consumed literally all available information on the planet - where do they go from here?