Since you’re not in a unique position, I can confidently state that your comparison of LLMs to jr developers seems unfounded. Today, LLMs produce code that is superior to junior developer code by an order of magnitude.
Notably, they demonstrate consistent syntax, clear separation of concerns, strong test coverage, organizational rigor, idiomatic API usage, and the ability to generate and maintain documentation, among other measurable qualities.
LLMs generally operate at a staff engineer level for a number of languages and ecosystems (including polyglot projects).
I'm not sure what your background is, but as a staff level engineer, I can assure you they do not. They in fact seem to lack any understanding of architectural intent within a sufficiently large code base. This seems obvious since they can't fit the entire code base in their context at once.
We have many folks (not engineers) at our company using LLMs to open PRs, and every one of these PRs has profound architectural design problems.
> They in fact seem to lack any understanding of architectural intent within a sufficiently large code base. This seems obvious since they can't fit the entire code base in their context at once.
Nonsense. The goalpost is "this is as good as a senior engineer". A senior engineer can easily understand architectural rationale. Don't dismiss my argument because it's inconvenient to yours.
LLMs absolutely do not exceed the abilities of junior devs. They don't even meet that bar, let alone exceed it. Junior devs are capable of getting syntax right without someone going "hey you messed that up". LLMs are not. Junior devs get basic logic right. LLMs do not.
Comparing an LLM to a senior developer is an absolute joke.
1. Which LLM are you using that is “not capable of getting syntax right”?
2. Are you referring to without having a compiler or LSP check it? Although even then, the recent LLMs I've used still frequently get syntax right, whereas I'd expect juniors are often using a LSP or compiler to catch mistakes while writing code?
who cares about syntax? who cares about iteration? what I care about are _results_, which they can produce at the end. do you check your human colleagues how many iterations they do before committing/showing their work to anybody? no. why should you set such a bar then for your LLM?
Since you’re not in a unique position, I can confidently state that your comparison of LLMs to jr developers seems unfounded. Today, LLMs produce code that is superior to junior developer code by an order of magnitude.
Notably, they demonstrate consistent syntax, clear separation of concerns, strong test coverage, organizational rigor, idiomatic API usage, and the ability to generate and maintain documentation, among other measurable qualities.
LLMs generally operate at a staff engineer level for a number of languages and ecosystems (including polyglot projects).