Interestingly it seems like we are investing many more magnitudes of capital for smaller and smaller gains.
For example, the jump in productivity from adding an operating system to a computer is orders of magnitude larger than adding an LLM to a web development process despite the LLM requiring infrastructure that cost tens of billions to create.
It seems that while tools are getting more and more sophisticated, they aren’t really resulting in much greater productivity. It all still seems to be resulting in software that solves the same problems as before. Whereas when html came around it opened up use cases that has never been seen before despite being a very simple abstraction layer by today’s standards.
Perhaps the opportunities are greatest when you are abstracting the layer that the fewest understand when LLMs seem to assume the opposite.
The real gains in software are still to be had by aggressively destroying incidental complexity. Most of the gunk in a web app doesn't absolutely need to exist, but we write it anyway. (Look at fasthtml for an alternate vision of building web apps.)
The issue with LLMs is they enshrine the status quo. I don't want ossified crappy software that's hard to work with. Frameworks and libraries should have to fight to justify their existence in the marketplace of ideas. Subverting this mechanism is how you ruin software construction.
You mentioned a great point that LLMs are hitting the edge of a marginal gain decreasing point, at least I think so. Many applications are struggling to provide real benefits instead of just entertaining people.
Another funny thing is that we are using LLM to replace creative professionals, but the real creativity is from human experience, perception and our connections, which are exactly missing from LLM.
As someone is not an artist I want ai to do art so I can restore my antique tractor. Of course we all have diffeent hobbies but there are also hobbies we don't want to get into but may nee.
I think the parent comment mean "art" as "having fun", like playing a guitar, definitely no fun to see the robot playing it and not letting you even touch it.
AI generated art/music/etc is the answer to people having creative vision and lacking technical expertise or resources to execute it. There are lots of stories waiting to be told if only the teller had technical ability/time/equipment to tell it. AI will help those stories be told in a palatable way.
Curation of content is also a problem, but if we can come up with better solutions there, generative AI will absolutely result in more and better content for everyone while enabling a new generation of creators.
The AI will also take over your work of restoring antique tractors, much faster and cheaper. It won't be historically accurate, and it may end up with the fuel pump connected to the radio but it'll look mostly Good Enough. The price of broken tractors will temporarily surge as they need them for training data.
If it can create some decal close enough where nobody know the original other than fragmets that remain that helps. For common tractors we know but I'm interested in thing where exactly one is known to exist in the world.
I see it very differently. We are just at the very dawn of how to apply LLMs to change how we work.
Writing dumb scripts that can call out to sophisticated LLMs to automate parts of processes is utterly game changing. I saved at least 200 hours of mundane work this week and it was trivial.
My favorite example of this is grep vs method references in IDEs. Method references are more exact, but grep is much simpler (to implement and to understand for the user).
I think you're also right about LLMs. I think path forward in programming is embracing more formal tools. Incidentally, search for method references is more formal than grepping - and that's probably why people prefer it.
For example, the jump in productivity from adding an operating system to a computer is orders of magnitude larger than adding an LLM to a web development process despite the LLM requiring infrastructure that cost tens of billions to create.
It seems that while tools are getting more and more sophisticated, they aren’t really resulting in much greater productivity. It all still seems to be resulting in software that solves the same problems as before. Whereas when html came around it opened up use cases that has never been seen before despite being a very simple abstraction layer by today’s standards.
Perhaps the opportunities are greatest when you are abstracting the layer that the fewest understand when LLMs seem to assume the opposite.