The number of famous CEOs who've hopped around from company to company despite being terrible at their job is probably in the dozens. The number of founders/CEOs whose companies have died is demonstrably in the millions.
Also, there are plenty of regular employees who suck at their jobs and yet manage to hold onto them, get promoted, get new and lucrative job offers, etc.
There are corners of the industry where people still write ASM by hand when necessary, but for the vast, vast majority it's neither necessary (because compilers are great) or worthwhile (because it's so time consuming).
Most code is written in high-level, interpreted languages with no particular attention paid to its performance characteristics. Despite the frustration of those of us who know better, businesses and users seem to choose velocity over quality pretty consistently.
LLM output is already good enough to produce working software that meets the stated requirements. The tooling used to work with them is improving rapidly. I think we're heading towards a world where actually inspecting and understanding the code is unusual (like looking at JVM/Python bytecode is today).
Future liabilities? Not any more than we're currently producing, but produced faster.
Compilers take a formal language and translate it to another formal language. In most cases there is no ambiguity, it’s deterministic, and most importantly it’s not chaotic.
That is changing one word in the source code doesn’t tend to produce a vastly different output, or changes to completely unrelated code.
Because the LLM is working from informal language, it is by necessity making thousands of small (and not so small) decisions about how to translate the prompt into code. There are far more decisions here than can reasonably fixed in tests/specs. So any changes to the prompt/spec is likely to result in unintended changes to observable behavior that users will notice and be confused by.
You’re right that programmers regularly churn out unoptimized code. But that’s very different than churning out a bubbling morass where ever little thing that isn’t bolted down is constantly changing.
The ambiguity in translation from prompt to code means that the code is still the spec and needs to be understood. Combine that with prompt instability and we’ll be stuck understanding code for the foreseeable future.
Humans are also non-deterministic, though. Why does replacing one non-deterministic actor with another matter here?
I'm not particularly swayed by arguments of consciousness, whether AI is currently capable of "thinking", etc. Those may matter right now... but how long will they continue to matter for the vast majority of use cases?
Generally speaking, my feeling is that most code doesn't need to be carefully-crafted. We have error budgets for a reason, and AI is just shifting how we allocate them. It's only in certain roles where small mistakes can end your company - think hedge funds, aerospace, etc. - where there's safety in the non-determinism argument. And I say this as someone who is not in one of those roles. I don't think my job is safe for more than a couple of years at this point.
> Generally speaking, my feeling is that most code doesn't need to be carefully-crafted. We have error budgets for a reason, and AI is just shifting how we allocate them. It's only in certain roles where small mistakes can end your company - think hedge funds, aerospace, etc. - where there's safety in the non-determinism argument.
That's a bit shortsighted. There have been cries of software becoming needlessly bloated and inefficient since computers have existed (Wirth, of course, but countless others too). Do you visit any gamer communities? They are constantly blaming careless waste of resources and lack of optimization in games for many AAA games performing badly in even state of the art hardware, or constantly requiring you to upgrade your gaming rig.
I don't think the only scenario is boring CRUD or line of business software, where indeed performance often doesn't matter, and most of it can now be written by an AI.
Even in CRUD line of business software, lack of performance causes enormous problems that the current software development culture glosses over.
Just one example I've seen time and again. You take an application that if optimized could run on a single server (maybe 2 if you absolutely have to have zero downtime deployments), but because no one cares about performance it runs on 10 or more. You now have a complexity avalanche that rapidly blows up. Then you need more hierarchy to handle the additional organizational complexity etc...
Then people start breaking out pieces of the app so they can scale them separately and before long you're looking at 200 engineers to do a job that certainly doesn't need that many people.
I realize I'm ignoring a whole lot of other issues that result in this kind of complexity, but lack of performance contributes to this a lot more than people want to admit.
Agreed. I wanted to give some credence to the fact many cookie-cutter CRUD apps can absorb a ton of inefficiencies until they truly burst at the seams, but yeah, even in that case software bloat and bad use of resources matters.
I find it intriguing seeing this new batch of dev-types completely giving up on the matter. The conversation of machine vs developer efficiency is not new, but completely giving up on any sane use of resources is something relatively new, I think. Especially coming from some in the HN crowd. Maybe these are new people, so I can chalk it up to generational turnover?
It has nothing to do with whether small mistakes are allowable or not. It’s about customers needing a consistent product.
The in-code tests and the expectations/assumptions about the product that your users have are wildly different. If you allow agents to make changes restricted only by those tests, they’re going to constantly make changes that break customer workflows and cause noticeable jank.
Right now agents do this at a rate far higher than humans. This is empirically demonstrable by the fact that an agent requires tests to keep from spinning out of control when writing more than a few thousand lines and a human does not. A human is capable of writing tens of thousands as of lines with no tests, using only reason and judgement. An agent is not.
They clearly lack the full capability of human reason, judgment, taste, and agency.
My suspicion is that something close enough to AGI that it can essentially do all white dollar jobs is required to solve this.
> adversarial AI reviewers, runtime tests (also by AI), or something else?
And spec management, change previews, feedback capture at runtime, skill libraries, project scaffolding, task scoping analysis, etc.
Right now this stuff is all rudimentary, DIY, or non-existent. As the more effective ways to use LLMs becomes clearer I expect we'll see far more polished, tightly-integrated tooling built to use LLMs in those ways.
Agents require tests to keep from spinning out of control when writing more than a few thousand lines, but we know that tests are wildly insufficient to describe the state of the actual code.
You are essentially saying that we should develop other methods of capturing the state of the program to prevent unintended changes.
However there’s no reason to believe that these other systems will be any easier to reason about than the code itself. If we had these other methods of ensuring that observerable behavior doesn’t change and they were substantially easier than reasoning about the code directly, they would be very useful for human developers as well.
The fact that we’ve not developed something like this in 75 years of writing programs, says it’s probably not as easy as you’re making it out.
> Agents require tests to keep from spinning out of control when writing more than a few thousand lines, but we know that tests are wildly insufficient to describe the state of the actual code.
Provide them with a mature, well-structured codebase to work within. Break the work down into tasks sized such that it's unlikely they'll spin out of control. Limit the scope/nature of changes such that they're changing one thing at a time rather than trying to one-shot huge programs. Use static analysis to identify affected user-facing flows and flag for human review. Provide the human-in-the-loop with fully functional before and after dev builds. Allow the human-in-the-loop to provide direct feedback within the dev build. Track the feedback the same way you track other changes. And, yes, have some automated tests that ensure core functionality matches requirements.
I think everything I've listed there can be built with existing technology.
> You are essentially saying that we should develop other methods of capturing the state of the program to prevent unintended changes.
I think you're imagining something far more sophisticated than what I'm actually suggesting. I also think you're setting a higher bar for agents to clear than what's actually required in practice.
Tests don't need to catch every issue, agents should be expected to make some mistakes (as humans do).
> However there’s no reason to believe that these other systems will be any easier to reason about than the code itself. If we had these other methods of ensuring that observerable behavior doesn’t change and they were substantially easier than reasoning about the code directly, they would be very useful for human developers as well.
There are lots of powerful static analysis tools out there than can be helpful in improving correctness and reducing the incidence of regressions. IME most human developers tend to eschew tools that are unfamiliar, have steep learning curves, or require extra effort when writing code.
> The fact that we’ve not developed something like this in 75 years of writing programs, says it’s probably not as easy as you’re making it out.
I think the cost/benefit of what I'm describing has changed. We've only had LLMs capable of reliably producing working code changes for around a year.
"users seem to choose velocity over quality pretty consistently"
When do they have a real choice, without vendor lock-in or other pressure?
Windows 11 is 4 years old but until a few months ago barely managed to overtake Windows 10. Despite upgrades that were only "by choice" in the most user hostile sense imaginable (those dark patterns were so misleading I know multiple people who didn't notice that they "agreed" to it, and as it pop ups repeatedly it only takes a single wrong click to mess up). It doesn't look like people are very excited about the "velocity".
In the gaming industry AAA titles being thrown on the market in an unfinished state tends to also not go over well with the users, but there they have more power to make a choice as the market is huge and games aren't necessary tools, and such games rarely recover after a failed launch.
> It's not like people can just decide to move to another country and they will say "sure, come on in!"
Many countries actively try to attract skilled migrants with simple, points-based immigration systems and fast processing times.
Simply having a bachelor's degree, 5+ years of work experience, and fluency in the local language will get you on the fast-track to a permanent working visa in many countries.
Back in the 2015-2020 self-driving investment boom I recall all kinds of dire predictions about the impact that robo-taxis and trucks would have on the US economy. There was an immediacy to these predictions. That once a viable driverless truck became widely available all truckers in the US would need to find new jobs the next day.
We can now see that it's going to take a lot longer for the technology to become viable and widespread. The impact is going to be felt in small pieces over many years, likely decades.
For a great many software projects no formal spec exists. The code is the spec, and it gets modified constantly based on user feedback and other requirements that often appear out of nowhere. For many projects, maybe ~80% of the thinking about how the software should work happens after some version of the software exists and is being used to do meaningful work.
Put another way, if you don't know what correct is before you start working then no tradeoff exists.
> Put another way, if you don't know what correct is before you start working then no tradeoff exists.
This goes out the window the first time you get real users, though. Hyrum's Law bites people all the time.
"What sorts of things can you build if you don't have long-term sneaky contracts and dependencies" is a really interesting question and has a HUGE pool of answers that used to be not worth the effort. But it's largely a different pool of software than the ones people get paid for today.
> For many projects, maybe ~80% of the thinking about how the software should work happens after some version of the software exists and is being used to do meaningful work.
Some version of the software exists and now that's your spec. If you don't have a formal copy of that and rigorous testing against that spec, you're gonna get mutations that change unintended things, not just improvements.
Users are generally ok with - or at least understanding of - intentional changes, but now people are talking about no-code-reading workflows, where you just let the agents rewrite stuff on the fly to build new things until all the tests pass again. The in-code tests and the expectations/assumptions about the product that your users have are likely wildly different - they always have been, and there's nothing inherent about LLM-generated code or about code test coverage percentages that change this.
You're not considering opportunity costs and buyers vs. users.
If your senior developers can slap together something better than an expensive SAAS offering you want them directing that energy at your core products/services rather than supporting tools.
And the people deciding to buy the expensive SAAS tools are often not the people using them, and typically don't care too much about how crappy the tool may or may not be for doing the job it's advertising as doing.
No matter what it's a tax on your engineering team to keep it together. But the most brittle parts are always right at the seams. It's not as hard to sew together components when you can cut the cloth down to fit together. Who knows how it'll shake out.
I hope your doctors clear you for exercise again and you feel comfortable giving it another go (after sufficient rest and recovery from this experience).
I had a similar - but less extreme - experience in my early 30s when I decided to start exercising after 20+ years of highly sedentary living. I somehow convinced myself that my heart rate being >160 while in "zone 2" was normal for me. In truth, what seemed like impossibly light exercise (5-10 minutes of "zone 2" every day) was too intense for me at the time. I burned out after about two months. It was very humbling to realise that the elderly people who jogged in the park near my apartment were more physically fit than me, and it took me a while to accept that.
When I eventually started exercising again I began with an intensity roughly equivalent to walking on a flat surface (HR around 105-115bpm) and stuck to a simple rule of thumb: if I didn't feel fully recovered 15 minutes after finishing a workout I had pushed myself too hard. From that baseline I was able to occasionally do a more intense effort, paying close attention to my heart rate during the workout, and being very mindful of what my body felt like during the effort as well as 15 minutes after, later that day, and the next morning. Over time I was able to ratchet up the intensity of 2-3 efforts per week and still feel fully recovered. After about a year I could do 3-4 genuinely hard workouts a week with a low risk of overtraining or burnout.
What I would point out is that in your original comment you said:
> I started running and cycling. Very short distances, very slowly and very gradually.
So from your perspective these were appropriate efforts. But then you go on to say:
> It made me feel horrible. Each time it would take me a few days to recover, feeling dizzy and mentally exhausted.
> A light jog would make my heart rate climb to 190 bpm immediately. A small couple of percent incline on a bicycle, straight to 190 bpm. Obviously feeling horrible afterwards.
These are really strong indicators that you were pushing far too hard.
That a light jog would be far too hard for a 34 year old is very confronting. It's a huge blow to the ego. I've been there, and worked my way out of the hole. Assuming there's no underlying medical condition I think you can too.
People who are extremely unfit tend to have no frame of reference for what a productive workout feels like. They are highly likely to push themselves too hard and then not give their bodies enough recovery time. Encouraging people in this situation to "push through it" is setting them up for overtraining leading to injury, illness, or burnout.
Fatigue is one of the most important signals your body can give you. It's a clear communication that you've been pushing too hard and need to reduce the intensity of your efforts. Telling people to ignore that signal for "at least a few weeks" is at best going to be counter-productive for them and, at worst, dangerous.
The person I replied to said they were taking it easy. By push through it I meant keep it up for a while longer even though it made them feel unwell, thinking they would get past the unwellness.
I wasn't intending to suggest anything remotely in the same ballpark as overtraining, that's not the kind of pushing I had in mind.
And when they clarified that they had been doing it for months I just said that's not normal, because a couple of months is beyond the scope I had in mind when I suggested pushing through it. It shouldn't take months. What they describe sounds more like a serious undiagnosed health issue.
> It made me feel horrible. Each time it would take me a few days to recover, feeling dizzy and mentally exhausted.
That doesn't happen with a genuinely easy effort.
> By push through it I meant keep it up for a while longer even though it made them feel unwell, thinking they would get past the unwellness.
Yes, that is really bad advice and will lead to overtraining in a very unfit person.
> I wasn't intending to suggest anything remotely in the same ballpark as overtraining, that's not the kind of pushing I had in mind.
I don't know what you mean by overtraining, but pushing through feelings of unwellness or fatigue and continuing to workout is exactly how you get into that territory.
They also said:
> Very short distances, very slowly and very gradually.
Which does not sound like overtraining to me. Any relatively healthy 30-something should handle that just fine no matter how untrained they are. They should get over the feelings of unwellness etc after a few weeks or at least a few months.
There is clearly some underlying condition causing this, it's not overtraining.
> stick with it for a few weeks before deciding whether you enjoy it
I think this can be bad advice for people who may be predisposed to dislike exercise due to past experience or just straight up unfamiliarity.
More constructive is to think about what you don't enjoy about the activity and whether those are intrinsic qualities or can be changed. If they can be changed, change them.
Similarly, it's a good idea to think about things you know you enjoy that can be coupled with exercise. Add those things to your exercise environment to build up positive associations over time.
As a simple example: if you feel self-conscious about your body when you go to the gym you shouldn't try to just suffer through that feeling. Just workout somewhere where you don't feel self-conscious.
Not true. Buffet's written a lot of great stuff on this subject.
reply