The other 11 items in the list of 12 would probably never be hit in practice, but now that you know, you have to sort out how to handle each (even if that just means documenting, filing, or ignoring: you have to understand each of the disparate 12 issues first.)
I suspect theres a lot of hours saved that are otherwise wasted by devs when some simple thing throws off their implementation completely. Basically we cant measure the saved developer hours by LLMs because it could be literally anything.
I think if there is a major change in code review culture, it's not going to happen first in established engineering teams. It'll happen in startups that adopt a no-human-review policy from jump and then maintain it as they mature. I'm sure that's happening, but it's likely invisible to most studies and surveys at the moment. If it proves successful, and such companies start shipping significantly faster, the rest of the industry will be forced to adopt.
i am pretty senior. I hate code reviews so much. They are so boring, easily the most tedious part of my job. Its such a chore. Who is clinging to line-by-line review. Ppl dont really give that much about their employers.
It really depends on the context. A react button background color - meh - the critical part of the backend where a new 300ms delay will cost the company millions then yeah.
these might be 1% typically
In this case, we're looking at a picture from 2025. The models involved in the main agent results were primarily Claude 3.5 through Sonnet 4.5, GPT-4.1 through GPT-5, and Gemini 2.5. And the paper's design means you're mostly looking at the very initial effects of deployment, before companies developed good review processes.
In any non-trivial integration test scenario, it's easy to end up several times the LoC in tests compared to actual shipped code.
Second problem: different models, agents, and harnesses seem to have different tendencies and they tend to avoid touching code that is not their own unless explicitly instructed. What I mean by this: it will happily duplicate some helper method from another test file rather than pulling it out and reusing it, seemingly avoiding coupling (good kind in this case since model level changes should be propagated atomically).
Tests are where I see agents just spew absolute mountains of code.
I think this approach will continue to work for some people and orgs, but others are going to hit walls where their agents can't manage the mountains anymore. It's going to be interesting to see how people and orgs respond when that happens.
> I've asked them what their plan is when 500 of their 1000 test suddenly stop passing
Bigger problems to me are more practical: 1) large tests waste context both in and out, 2) large volume of tests waste CI cycles which are becoming the bottleneck.
Each time we move up an abstraction layer, people object, saying that nothing will work right if we move up that high in the stack. Give it time. I'll bet in 5-10 years, almost no one will look at code anymore, except in special cases.
We still have people who write in assembly, we still have people who write in C, mostly for performance reasons. Because you still need to be close to the CPU to get that last bit of performance. But in most cases, it doesn't matter. It doesn't matter if the LLM makes bad code if it works and passes all the tests.
Granted, this is the first time we're going from deterministic to non-deterministic, but is it really? Large scale distributed systems are already non-deterministic. They create inputs and situations that programs aren't designed to handle all the time. What's a little more chaos? :)
- the data is mostly from 2025 and earlier
- the baseline mean lines of code per month was 5017, the estimated increase was 1494 lines per worker-month
- the baseline is 5.34 PRs per month, and increased by 1.22
- the baseline mean for code review is 7.03 days
The second and third bullets are confirmation that this is old data. In late 2026, people aren't delivering 30% more lines of code. If people were just producing 30% more lines, they'd still be reading every line. My point isn't that "more lines is good". I'm worried about a few too many lines of code that no one reads. My point is that this shows the data is old.
For the latter, I'm a bit annoyed by using mean, median would be more valuable, but I think it's safe to say that the engineering culture was already pretty bad. If your average pull request sits for 7 days, you evidently don't care about shipping anything, and agents aren't going to change that.
yolo works when I need to focus on higher priority, and make sure things just work (though not "great").
For the latter, I use it for more like spec-driven fashion with HITL. Basically everytime it ffinishes a task, I review and then move on.
Both appraoches work. just make sure your yolo slop isn't bad and passes stuff.
LoC don't mean jack shit.
I generate more beautiful code, yes.
But the longer the codebase survives the more hair and cruft it accumulates. My once beautiful logical paths get carved up by compounding requirements and edge cases.
That's what the AI was trained on. The end result - the hairy ball of wax that noone can look at sideways.
I often think of that whenever AI comes up with what I consider to be an irrational but plausible edge case.
I never have operated this way. Code reviews were a basic sanity check, a chance to implement best practices, and a chance to teach these things to juniors.
If I was concerned about something breaking, I'd duscuss test coverage.
Was I doing it wrong this whole time??
If it is so critical, why don't we have tests and benchmarks for it?
Oh right, because "Greg the super-coder" never wrote any of this down and it lives in his head so he can be the arbiter of change.
It was toxic before agentic coding, it's still toxic now.
Document your systems. Add tests and benchmarks for what is important.
We've always known this, we have a dozen or more textbooks written about it. But we always gave it a pass because "Greg the super-coder" was producing more value than he was hurting.
That's no longer the case. Your best coder is now Opus or Astra
I’ve seen a few projects that heavily rely on ai with not very good (though senior) devs driving it - and its a mess. Not the code. The resulting product. Code is actually sound as ai is pretty good now.
All because ai does code reviews and dump decisions go into prod
I had it do a full audit. There were about 100k lines of tests, of which it determined about 60k were near useless. Many were overly mocked, some mocked to the point that they really weren't testing anything at all. It deleted those and wrote more expansive scenario tests to replace them, caught a lot of bugs on the way and fixed those too. Total time was about 2 hours.
I think humans need to review requirements, specifications, and test scenarios. And for some industries and uses they're still going to have to review every detail. But for a typical CRUD app, I think we're at a point where we can already set the process and quality bar higher than ever, in addition to moving faster.
My company's leadership is oscillating every few weeks on AI goals. Their most recent story is "we're expecting developer velocity to approximately double over the new 2-3 years as people get better at using AI". But of course that's completely non sequitur. People on my team spend about 10% of their time writing code and 90% doing design and analysis. So if AI doubles our code writing velocity, then it will make us 5% faster overall.
But how many changes are actually serious?
I can have an agent mass-produce security bug fixes, zip through our low-priority backlog, etc.
IMO - we need to get out of our own way.
Building is cheap now. Agents are also really good at following checklists and running tests.
So instead of having individual engineers reviewing code, we need to
* Leave human review for the most impactful changes
* Get better at defining the quality bar
* Continually codify human concerns into rules or systems that can be checked against
A lot of this was touched on a few months ago with Spec-driven development becoming a hot topic when agentic coding started.
We should continue that with spec-driven review, testing, etc.
If you take it off too early you end up with shit and will spend weeks fixing all these baked bad assumptions your project is built on top of. I spent weeks fixing some of this shit then the LLM revived a not insignificant portion of it in a later change because it got "confused" (its words) because some of the comments it previously made were left around.
I actually dislike traditional code review. In part this is because most of our code review tools are not very good to begin with (GitHub's and GitLab's web interfaces in particular).
But also because humans aren't all that great at code review when it comes to correctness and security. Part of the problem is that you don't have access to the author's mental model for non-trivial changes and have to reconstruct it, otherwise your understanding of the program logic will be inadequate.
The problem is that purely agentic code review has orthogonal problems, and typically remains subpar when it comes to judging architecture and design considerations, compared to human reviewers. Both false positives and false negatives also remain a concern.
My best results so far have been in agent-assisted code review, helping you to quickly construct a mental model and to analyze the underlying logic (in addition to doing a first pass for correctness issues).
alexnewman•59m ago