https://github.com/jacquardlabs/gauntlet

I'm concerned that there are people pushing on the code generation front, the code reviewing front, and the laziness front. Eventually they're going to all meet in the middle, and there will be a significant number of engineers who are using AI to write and review the code, rubber stamp it, and push it into prod, with disasterous results.

Day Job: Similar but more human-in-the-loop and we are still feeling out "what needs human review" and "agent review is good enough".

LLM reviews have shown me/us that they will catch more things (legit things and silly things, to be fair) than human reviewers. Human reviewers are very bad at seeing what's _not_ there. Yes, humans will catch some things that the LLM won't but it's normally only on very custom things we've done that the LLM isn't "trained" on. [0]

I think that as time goes on code reviews will be almost fully automated and humans will focus more on the plans before building, overviews of what was built, and other spec/diagram-type "artifacts" than looking at the actual code.

[0] For my side projects and even a little for work I've been trending in the "do it the way the LLM wants" not "force it into the shape you thought of" because it's easier for the LLM to write/maintain/"understand". Similar to how I don't tell an LLM "put a red button on the page here that does X", I give it the problem I'm trying to solve so that I don't "bias" it to the way I first thought of. I may force it to do it "my way" in the end but I find I get exposed to new ideas or new/different ways to solve my problem when I don't "lead the witness".