This is a pretty good middle ground, I think. You can't prevent LLM usage and there's significant downsides to doing so universally, so restricting contributions to things that a human needs to demonstrably understand circumvents a lot of problems.
I think this should be combined with banning people who cheat by trying to explain code without properly reviewing it and burning cycles from humans at the other side.
At least that would be my policy if AI is allowed.
I find myself often trying to understand the madness of some of the code an LLM produces. Does that count? You're absolutely right! I made a mistake, and I'm sorry.. Then I end up questioning myself if it wouldn't have been easier to just do it myself. To be fair though, it's not limited to LLMs, I've felt that way other people's code too.
Jokes aside, there's a difference to understanding the code and understanding the reasoning that is behind the code, I feel that LLMs still struggle enormously with the later. They start writing, and sometimes realise halfway through that they can't backtrack and just keep writing rubbish. You can argue about spinning loops and iterative processes, as long as they are actually able to converge.
I noticed something about LLM code generation and this is as good as place to post the finding as any.
It's fairly obvious, but as we know LLMs choose the highest probability next token, so when generating code, it does so from left to right, without ever reorganizing, which unless it uses a harness, that's not how WE write code, that's property 1.
Property 2 is that it will write out as many boilerplate that occurs before the actual implementation as it can, because boilerplate is always the same, and implementation is high temperature/chaotic, there's many different ways an implementation can go. For example in python, you can write your code directly, or wrap it in a main loop and later add a main guard. LLMs will always write the main loop, since it's not competing with NOT writing a main loop, it's competing with the best option in the set of non-main loop solutions. This is trivial in this case because all non main solutions are present in the main loop solution set, but for solutions where the two sets are distinct, the LLM will have a bias towards solutions that share initial tokens, e.g:
Solution 1: import lib1 and use function A 30%
Solution 2: import lib1 and use function B 25%
Solution 3: import lib2 and use function A 45%
Despite solution 3 being weighted more heavily, the LLM will opt for solution 1, since solution 2 makes it choose the import lib1 token.
This pushes towards mega-libraries instead of composable Unix libraries. Stuff like numpy, react or helper libraries get a boost since they are more like megaframeworks than specific libraries, and they get their import statements boosted.
I think I get your point. I can't say that I've observed it writing boilerplate just for the sake of it, but it is disproportionately/abundantly present in the training set. One of the first rules they teach in ML is to balance datasets, even if it's with fake/interpolated samples.
The linear L->R generation is definitely a thing, it's much more costly for an LLM to iterate edits, where a skilled vim coder will be jumping all over the place, trying to make all the LEGO pieces fit.
The skill therefore relies on just being able to one-shot entire chunks of code correctly, and it's amazingly good at this... But even the SOTA models still have a lot of unused imports and unused variable declarations. They just have to "guess" what they'll need and hope for the best. If they include a mass of numpy/scipi/react/icon imports that they might need, it opens the landscape for them later on when predicting relevant tokens, reaching a more ideal solution.
It doesn't hurt to add imports that might be helpful, rather than penalize the solution because you haven't got them. Although the last few SOTA models are more "harness/tool aware", they're starting to have the instinct to write the code anyway, and to be allowed to go and fix the imports later via tool calls.
For anyone who's seen the film Arrival (2016), their entire language is formed of complete concepts, not sequences of words and time. I keep thinking back to this.
Sorry for skipping over your actual argument, but if it hinges on that assumption then it's probably moot. I'd assume almost all ai generated code that makes it into codebases is produced using a harness.
if it uses a harness, the effect is still there, just piled on and magnified.
I sometimes gen code without a harness and copy paste it or manually type it, maybe I can do like 200 lines in a day? Whenever I see someone coding with a harness it's like 100x times that, so this phenomenon will happen hundreds times more.
