There's obviously some nuance here, but the fact is that much modern software is riddled with bugs, and this is sub-optimal for everyone (both software users and software builders). Most of the bugs which frustrate and irritate software users are not due to uncontrollable events such as cosmic rays flipping a bit. Most of them are plain old code defects.
But, you do have a valid point. Allow me to rephrase it this way: The answer is not for software companies to spend unbounded amounts of engineer time chasing every reported bug.
But there are ways that we, as an industry, can do better, and it's not by pouring all our time into chasing hard-to-diagnose bugs. Here are a few ways that I personally see:
1. Some very powerful technologies for finding many bugs with little engineering effort already exist, but are not widely used. As an example, coverage-guided fuzzing is amazingly good at finding all kinds of obscure bugs. The idea of coverage-guided fuzzing was known from the 1990's, but it took AFL (in ~2013) to take it mainstream. Even now, much of the industry is not benefiting from the awesome power of coverage-guided fuzzing. And there are other, equally powerful techniques which have been known for a long time, but are even less accessible to most software developers.
So: spread the word about such techniques, and for programming language/platform developers, work on making them more easily applicable. This could help many software companies to catch a great number of bugs before they ever go to production.
2. Similarly, there are extant historical computing systems which had very powerful debugging facilities, much better than what is currently available to most developers. The ideas on how to make our platforms more debuggable are already out there; it's now a matter of popularizing those ideas and making them readily accessible and applicable.
3. Since it's widely known that many bugs (real bugs, not "cosmic rays") are extremely hard to reproduce, an admirable target for us to aim for as developers is to implement debug logging in a way which allows us to root-cause most obscure bugs just by examining the logs (i.e. no need to search for a reproducer). Some real-world systems have achieved that goal, with very good results.
4. While there is currently much buzz about using LLM-based coding agents to write code, I think an almost better use case for coding agents is in triaging bug reports, diagnosing the bugs, finding reproducers, etc.
I've recently had a couple of shocking experiences where, just given a written description of an intermittent, hard-to-diagnose bug, a coding agent was able to search an entire codebase, identify the exact cause, and write a reproducer test case. (And this after multiple experienced human programmers had looked at the issue but failed to identify the cause.)
In summary, I think there are ways to "cut the Gordian knot" of bug reports.
Yes, quite! Monsieur Bellard is a legend of computer programming. It would be hard to think of another programmer whose body of public work is more impressive than FB.
Unfortunate that he doesn't seem to write publicly about how he thinks about software. I've never seen him as a guest on any podcast either.
I have long wondered who the "Charlie Gordon" who seems to collaborate with him on everything is. Googling the name brings up a young ballet dancer from England, but I doubt that's the person in question.
> It would be hard to think of another programmer whose body of public work is more impressive than FB.
I am of the firm belief that "Monsieur Fabrice Bellard" is not one person but a group of programmers writing under this nom de plume like "Nicolas Bourbaki" was in Mathematics ;-)
I don't know of any other programmer who has similar breadth and depth in so many varied domains. Just look at his website - https://bellard.org/ and https://en.wikipedia.org/wiki/Fabrice_Bellard No self-aggrandizing stuff etc. but only tech. He is an ideal for all of us to strive for.
Watson's comment on how Sherlock Holmes made him feel can be rephrased in this context as;
"I trust that I am not more dense than my neighbours [i.e. fellow programmers], but I was [and am] always oppressed with a sense of my own stupidity in my dealings with [the works of Fabrice Bellard]."
Lua isn't my primary programming language now, but it was for a while. My personal experience on the library ecosystem was:
It's definitely smaller than many languages, and this is something to consider before selecting Lua for a project. But, on the positive side: With some 'other' languages I might find 5 or 10 libraries all doing more or less the same thing, many of them bloated and over-engineered. But with Lua I would often find just one library available, and it would be small and clean enough that I could easily read through its source code and know exactly how it worked.
Another nice thing about Lua when run on LuaJIT: extremely high CPU performance for a scripting language.
In summary: A better choice than it might appear at first, but with trade-offs which need serious consideration.
Be tactful and kind, but straightforward about what you can't/don't want to spend time reviewing.
"Thanks for the effort, but my time and energy is limited and I can't practically review this much code, so I'm closing this PR. We are interested in performance improvements, so you are welcome to pick out your #1 best idea for performance improvement, discuss it with the maintainers via ..., and then (possibly) open a focused PR which implements that improvement only."
Depends on context of course, but in my book "my time and energy is limited" is not a valid reason for a reject. Get back once you have time, review in chunks.
ivanjermakov, I don't know if you are an open source maintainer or not (I am, for several projects). If you are, and you follow the policy that "I will never reject PRs because of having no time, I will always get to it eventually", then I salute you. That is a self-sacrificing, altruistic position to take. It's also a very difficult position to maintain for the long term. If you can do it: congratulations!
As for me, my position is: "My project is my house. You want to be a guest in my house, you follow my rules. I really like people and am usually happy to answer questions from people who are reasonably polite, to review and provide feedback on their PRs, and so on. But I won't be pressured to prioritize your GitHub issue or PR over my work, my family, my friends, my health, or my personal goals in life. If you try to force me, I'll block you and there will be no further interaction."
If you don't like that position, well, I understand your feelings.
I'm absolutely with you on that. I'm not saying that every contribution deserves equal attention and that rejecting contributions is a bad/impolite thing.
There has to be a better reason than "your PR is too big" as it's likely just a symptom, also very much context sensitive. If it is a 5kLOC PR that adds a compiler backend for a new architecture then it probably deserves attention because of its significance.
But if it's obviously low quality code than my response would be that it is low quality code. Long story short, it's you (submitter) problem, not me (reviewer, BDFL) problem.
My belief is that whatever technology can be invented by humans (under the constraints of the laws of physics, etc) will eventually be invented. I don't have a strong argument for this; it's just what makes sense to me.
If true, then an immediate corollary is that if it is possible for humans to create LLMs (or other AI systems) which can program, or do some other tasks, better than humans can, that will happen. Inevitabilism? I don't think so.
If that comes to pass, then what people will do with that technology, and what will change as a result, will be up to the people who are alive at the time. But not creating the technology is not an option, if it's within the realm of what humans can possibly create.
I think you do. Have we ever been successful at slowing down technological efficiency?
>If that comes to pass, then what people will do with that technology, and what will change as a result, will be up to the people who are alive at the time.
If it is inevitable that technology will be developed, it is also inevitable that it will be used, and in turn, further technology developed. Technology is an arms race. You can't opt out once you've started. If you do not employ the same technical progress for whatever-- propaganda, profits-- you will lose.
I know you're not posing it as a problem or solution, but I believe pinning it completely on "it's how we use it" is not a valid tactic either.
“Have we ever been successful at slowing down technological efficiency?”
Yes, we slow down technological efficiency all the time. Nuclear Power for one. I think you could argue we did the same for blockchain, once the hype died down. I might argue most technologies we slow down by divesting from them as their core use cases subside.
Facebook has been pivoting away from the metaverse which means we’re slowing down research in that area.
There are absolutely times when one can get LLMs to "say something valuable". I am still learning how to put them to good use, but here are some areas where I have found clear wins:
* Super-powered thesaurus
A traditional thesaurus can only take a word and provide alternative words; with an LLM, you can take a whole phrase or sentence and say: "give me more ways to express the same idea".
I have done this occasionally when writing, and the results were great. No, I do not blindly cut-and-paste LLM output, and would never do so. But when I am struggling to phrase something just right, often the LLM will come up with a sentence which is close, and which I can tweak to get it exactly the way I want.
* Explaining a step in a mathematical proof.
When reading mathematical research papers or textbooks, I often find myself stuck at some point in a proof, not able to see how one step follows from the previous ones. Asking an LLM to explain can be a great way to get unstuck.
When doing so, you absolutely cannot take whatever the LLM says as 'gospel'. They can and will get confused and say illogical things. But if you call the LLM out on its nonsense, it can often correct itself and come up with a better explanation. Even if it doesn't get all the way to the right answer, as long as it gets close enough to give me the flash of inspiration I needed, that's enough for me.
* Super-powered programming language reference manual
I have written computer software in more than 20 programming languages, and can't remember all the standard library functions in each language, what the order of parameters are, and so on.
There are definitely times when going to a manpage or reference manual is better. But there are also times when asking an LLM is better.
If your firm has been around for long enough, then it will almost certainly have long-standing clients who repeatedly come back for changes, additions, and general maintenance to existing software, as well as new development. Those long-standing clients will also introduce new clients.
You cannot control when a customer buys and you cannot create a sale.
All you can do is charge enough to stay in business and try to be worth your customers wanting to keep you in business.
Nobody cares if you start a business; you will be competing against companies whose customers pull their bootstraps; and starting out you will mostly deal with the inexperienced and the grifters.
But, you do have a valid point. Allow me to rephrase it this way: The answer is not for software companies to spend unbounded amounts of engineer time chasing every reported bug.
But there are ways that we, as an industry, can do better, and it's not by pouring all our time into chasing hard-to-diagnose bugs. Here are a few ways that I personally see:
1. Some very powerful technologies for finding many bugs with little engineering effort already exist, but are not widely used. As an example, coverage-guided fuzzing is amazingly good at finding all kinds of obscure bugs. The idea of coverage-guided fuzzing was known from the 1990's, but it took AFL (in ~2013) to take it mainstream. Even now, much of the industry is not benefiting from the awesome power of coverage-guided fuzzing. And there are other, equally powerful techniques which have been known for a long time, but are even less accessible to most software developers.
So: spread the word about such techniques, and for programming language/platform developers, work on making them more easily applicable. This could help many software companies to catch a great number of bugs before they ever go to production.
2. Similarly, there are extant historical computing systems which had very powerful debugging facilities, much better than what is currently available to most developers. The ideas on how to make our platforms more debuggable are already out there; it's now a matter of popularizing those ideas and making them readily accessible and applicable.
3. Since it's widely known that many bugs (real bugs, not "cosmic rays") are extremely hard to reproduce, an admirable target for us to aim for as developers is to implement debug logging in a way which allows us to root-cause most obscure bugs just by examining the logs (i.e. no need to search for a reproducer). Some real-world systems have achieved that goal, with very good results.
4. While there is currently much buzz about using LLM-based coding agents to write code, I think an almost better use case for coding agents is in triaging bug reports, diagnosing the bugs, finding reproducers, etc.
I've recently had a couple of shocking experiences where, just given a written description of an intermittent, hard-to-diagnose bug, a coding agent was able to search an entire codebase, identify the exact cause, and write a reproducer test case. (And this after multiple experienced human programmers had looked at the issue but failed to identify the cause.)
In summary, I think there are ways to "cut the Gordian knot" of bug reports.