When I got into agentic coding a year or two ago I was sure it was only good at autocomplete. Something happened earlier this year where the models hit a new level of capability.
Everyone I know now just does agentic coding, and it’s really amazing. I think we should just try pushing this as far as we can possibly go, it really feels like the acceleration of the human race is upon us.
Besides, I have been hearing "this is the limit" since the doomers of "this is just a markov chain and can't be useful".
Yet the limits keep being broken.
Yes, something happened, it got better at autocomplete. What else could be? The underlying model hasn't changed.
>acceleration of the human race
Please just stop with this bullshit. Nobody's curing cancer, climate change, inequality or whatever important real problem there is with LLMs. Nobody.
If this tech is good enough to make you more productive is just because you're not working in anything new or cutting edge or innovative. The only reason a LLM knows how to do your job is because that code has been literally written before enough times to appear in the training data. Try to use llms to write C++26, some HDL or in any niche stack and you'll get a nice reality check about LLMs.
Are we really still doing this?
Is this any different than how a PM gets a programmer to work on a project? They think, then they deliver. If given more time, maybe they deliver something better. Maybe they consult some text and try to apply a design pattern.
The LLM in this use case is perfect because almost everything involved is text based, and the model is able to take in all the expressive that is language.
Yes, it's very different. You seem to be suggesting that the current frontier LLMs, when tied to their tools and harnesses, have emergent properties that are similar to human consciousness. If you truly believe that, I'm not sure how to have a productive discussion here.
Its fundamentally how LLMs work.
However there's still a distinction. Unless I'm responding to an LLM, you had a childhood. You learned about the world and space and agency before you ever learned how to program. And you didn't learn it from billions of examples, you learned from a few examples, some self directed experiments, some feedback from teachers, etc...
I'm saying that's what matters. The process matters. You didn't learn to mimic a distribution, you learned to program. Of course in the perfect mathematical limit it's the same, but in practice it's not.
Agents code extremely well.
They're not particularly good at 'architecture' and I think that's where his specific concerns about 'not being able to see the problems' arise - the issues are are almost never in the syntax, because the AI writes perfect code. The issue is that it's not doing exactly what you intended.
Instead of 'missing the target' ... it's 'hit the wrong target perfectly'.
Any senior developer working with AI daily should be able to have a baseline intuition for all of this, and would therefore reject the hyperbole of the premise 'it can't code!'.
Of course it's producing gargantuan amounts of slop - that's not because 'it can't code', that's something else entirely.
Which has a good track record of being right, most of the time.
I agree!
Could AI technology change the world? Sure. Will it? That depends on so much more than what the technology can do. Why are we all still working 40 hours a week? Why are people still hungry? We could have radically changed our world with the technology we have had for decades. Yet, we have not, we have continued, nothing has really changed.
The internet is a great example. What is the most impactful part of the internet today? Social media. Social media has radically changed our culture. What is social media? A database, a few endpoints and an app? The technology is the least consequential part, the consequence comes from how we use it.
Nerds focus on what is possible with the technology, not what society is likely to do with it. What evidence is there that AI is going to change the world? What change is going to come from... being able to generate plausible sounding text? From being able to instruct agents? How many companies are using garbage software from 20 years ago despite dozens of revolutionarily better equivalents being available out there today that could have drastically reshaped their workforce? What are agents if not better macros? How many businesses have hundreds of employees doing the same tasks over and over again that could have been replaced by a few macros? How much of the code that you and I have written in our careers has already been written before?
The fundamental usefulness is the least important part of a technology when discussing how that technology will impact the world.
I don't understand how it's remotely reasonable to try to make the comparison.
I always wonder whether HN suffers from periodic influxes of newbies who don't get it yet and rile up the regulars.
My main issue has been the inconsistent quality across between model releases and the tendency to insert older APIs or documentation, especially with command line tools.
I can understand if the model struggles with a million line monolithic codebase with a decade of cruft but can't think of why it'd be too much of a pain with new codebases.
I’m a little more hopeful than the author though. I feel like it’s possible to manage the process so that does not happen.
I've also managed to use LLMs to cut a lot of manual duplication in code where we typically didn't do enough investment: "Claude, evaluate code duplication in the functional test suite" will have no problem finding things like insufficient helpers, or tests that are testing simpler things as prerequisites, so they can rely on each other. So I am not seeing my codebases growing all that much. There's some risks of functional changes that before would be rejected due to cost which now are not, but I am not all that sure of how much that is controllable without being relatively antagonistic with management.
Seems like it is completely hopeless at doing anything netcode consistency and performance related in game dev. Seems like unique game mechanics it doesn't do well either.
Seems like asking it specific UI stylistic changes is basically like throwing darts at a board and hoping it sticks.
Ah I see your org hasnt yet had an outage caused by a bad LLM code push.
https://blog.k10s.dev/im-going-back-to-writing-code-by-hand/
Programmers are rewriting and reinventing the same techniques more often than any other vocation I can think of, and so we were primed for a really good search over prior art. The fact that AI can also adapt that prior art to your particular use case makes it even more powerful.
Much like how great success never came from cobbling together various bits of copy-pasted code from Stack Overflow though, current AI can't really build your whole project.
Well that's what everyone is claiming anyway
If I were to use it against a legacy, rather poorly written codebase, where the code may be hard to understand without some in-depth analysis. I could certainly ask an AI agent to read the code (How does application X do Y, for example), but I wouldn't have it start hammering out features or have it do any type of refactoring. That would cause far too many commits and confusion amongst the development team, leading to even more slop than whatever we'd already be dealing with.
Just leaving this comment here so I can come back to your comment. I've been getting a bit discouraged by AI lately, but this sums up my experience with it well enough.
We're currently using it to build out a full-scale application. It does as well as you care to coax into doing tbh. You have to invest heavily in harness engineering, and at least my experience has been that as you do that, the results improve.
That is also my experience.
When starting a project I observe how the agent fails, I add new rules to the harness to prevent it from falling and repeat the process until I am happy with the output.
EDIT: To people downvoting me, please come up with a reasonable bet and lets try to work it out.
EDIT 2: $500 bet paid to your account on whether LLM's are going to still be used productively or not. No one?
EDIT 3: Any bet that would express the author's argument in a way that can be disproven in the future
To use a Geohot-inspired analogy, what we have now is like the Google self-driving car of 2010. It works most of the time, yet sometimes fails in unpredictable ways. So you need a safety driver behind the wheel to constantly watch what it's doing (the code review).
A real AI agent would not need a safety driver. We don't have that but many people are basically saying "fuck it, I'm just going to set this car off on its own and see what happens". And sure if you're prototyping it's not dangerous. But for production systems that is dangerous.
There is some very cool tech it just needs continued refinement, there is a path forwards even if it isn't always the clearest. This is happening but it is taking years and a lot of work to get done.
In other words - they can program, and probably better than you.
I don't like being too critical but this is a really superficial post - as if either 'AI is a Software Engineer - or - It must be Fraud'
It's an extremely powerful tool that is very 'pattern oriented' and with guidance can absolutely write great code - and even across modules given the right basis.
It's also great at so many other tasks - finding bugs in big code bases, doing migrations etc.
It's not going to make very goo architectural decisions for you, and if you're doing anything novel you have to read most of the code ... but that's too be expected.
https://en.wikipedia.org/wiki/George_Hotz
In fact, he’s done several things that are truly hard, and has a well-deserved engineering reputation.
It's ridiculous to suggest that 'AI can't code' - when the entire development world has moved into agentic coding, including all of the best developers in the world, and it's yielding positive results in most scenarios.
It's a callow 'bad twitter take' the length of an article.
He's not wrong to suggest that IA is a 'stochastic mechanism' over all the code that's ever been written, but that's evidence of the mechanism, and frankly, describe how it is able to code.
And yes - organizations will misappropriate AI at scale as they do with everything.
His premise is so far out of proportion and misguided, it's tantamount to 'fake moon landing' conspiracy theory.
Careful, your bubble is showing.
-- I think this article is COPE, if I'm being quite honest. I thought of putting cute analogies, like the C programmers saying the Python and Javascript programmers are not "hardcore" enough... but the truth should be obvious to anyone using LLMs effectively.
-- Current AI is a much better programmer than 100% of people and when directed by someone in that top 10%, it's a force majeur.
You guide the AI with some prompts and give it some guidance on how to scenario-test it. It makes some classes, test methods. Maybe ~2000 lines and you do a quick verification, check if the overall idea looks okay. Ask it to fix a few design things and then merge it.
Its much easier than doing it yourself with all the boilerplate and understanding each esoteric language specific thing. Which library do I use for UDP communication in golang? The agent might have made a good assumption. These kind of things is where it speeds it up.
LLM's are directionally right and if their answer "fits" then I take it at face value.
I wrote a blog detailing the computational difference between "generation" and "verification" and why it matters for LLM's: https://simianwords.bearblog.dev/the-generation-vs-verificat...
As an example: I asked the LLM "synonym for "provides" that also means "places" on you" and it gave me 5 answers and I immediately knew the right one was "confers". How? It just fits. Just like most things.
We have our core code in a weird dialect of C and rust. C I know well, but not rust. Our tests are in Python. The pipeline descriptions are in Yaml.
Outside of the core code there are so many arcana to learn. Writing syntactically and semantically correct yaml/Python test code would be a nightmare. The Agents have flaws, but they provide a huge leg up in improving the tests.
And they are great at providing a first pass review of the core code before bothering a human reviewer. Lastly we run some of our test failures through AI triage, which often enough finds the root cause or rules out simple failures.
This shows up in a higher checkin rate. I'm curious to see whether this will lead to quality end product since we have more support for the more manually written and reviewed core product code.
But it can write working code much faster than I can.
And in a lot of cases, unfortunately, faster beats better.
Agents now are writing extremely consistent, normalized canonical code, that usually compiles the first time.
Right out of the 'textbook'.
For what it's trying to do - it writes nearly perfect code.
The only thing you could nominally disagree with are some of the conventions and idioms.
It 'writes a perfect novel, in perfect prose'.
What it will not do however, is 'write the novel that's in your head'.
And that's the crux of it.
It's not even your job to 'write code' at this point, but rather to be the storyteller - and a very good editor who has enough taste and grasp of gammar to be able to know when it's going awry.
It will make mostly what you tell it too, the quality of the output is the quality of your guidance, but at the lowest levels it's generating extremely high quality syntactic prose.
Modelling a problem is what I'm concerned about. And I'm currently better than any AI agent at doing that, given enough time.
Those matrix multiplications aren't a divine perfect thing. They suffer from floating point precision issues and training data issues and there's still debate if adversarial examples are just an unsolveable property of our linear-algebra based neural network architecture.
Can they do things way faster than a human? No doubt. Can they do very complex tasks? Yes. Do they do things with perfection? Not by our human definition of perfect.
I think you have just written the epitaph for corporate software.
I hate how both the for and against case for LLMs are just so bloody terrible at addressing these things.
You have to use something like superpowers, the key is that the humans need to make the important decisions.
You have to review the code - just like you had to review the code humans wrote. There will be iterations.
You have to give the LLM skills and patterns to follow, access to architectural documents, etc, just like humans needed to be onboarded at a company and do the same.
If you get all of these right with today's LLMs, you will never write code at all because it is so obviously not the best use of your time. If you feel that you are still better at writing the code manually, you have not done the above right, fix your workflow and try again.
For something to take "longer and longer" to realise, doesn't they imply that it's been realised at least once before or that there was an expected deadline for the realisation?
Okay, that's a nitpick.
I hope that professionalism still matters as these new ways of doing things strikes me as unprofessional as f...
Yeah, the next macOS will be worse... time to place bet on prediction market
If your work was previously googling stack overflow, it can be incredibly useful at working through that. Which let’s face it, that’s what a lot of us do.
At the same time in no part of his post is any code snippet or anything to latch on to of "the model performed poorly here when it should have done this" - this style of criticism seems to be a pattern of most of these "the LLMs will never work" style posts on blogs and twitter.
They obviously can perform better than autocomplete and in my own day to day development build out huge portions of a codebase that I would have expected a junior or midlevel engineer to perform at.
How are we really supposed to grasp their actual capabilities when no one will actually cite specifically what mistakes they are making.
The mistakes they make are pretty subtle. Coding with LLMs can be like that scene in Whiplash – <excellent drumming >, not quite my tempo, <excellent drumming >, downbeat on 18, <excellent drumming>, you’re rushing, <excellent drumming>, dragging, …
Like yeah it produces working code almost always and the code usually does what you asked. And yet it makes you want to throw a chair because it’s not quite right in frustrating ways and it doesn’t even have the taste to know how it’s wrong.
I have no doubt the top nth percent of coders could write circles around Claude or Codex, but how much worse are they than your average schnook?
So what he is telling us? That agents are not infaillable and they are not capable to one shot complex software and they do not produce perfect code?
We know what and the solution is to use agents for what they are good at and work around their limitations and we have a human in the loop.
>not some RLVR shit that comments out the failing test and tells you all the tests are now passing
That's what harnesses should be about: detect when the agent is misbehaving and force it to take the right approach.
This example in particular should be easy to solve if we generated the tests before coding and we have a workflow or state machine that doesn't allow the agent to disable tests and doesn't allow it to reach the next stage unless all tests are passing.
I mean, this has been the trend for decades really, before LLMs were a thing. The incentive is skewed toward quantity rather than quality. The new tools just add more fuel to the fire.
Code quality is also really lacking in much of the industry. The truth is, these LLM models, as limited as they are, program at a level above that of the median junior programmer.
If you are already comfortable with letting other devs work on features then it's easier, because it's similar (arguably you have more control with AI, because what you say goes regardless of hierarchy).
If the answer is yes, the argument doesn’t matter: you just run the loop and wait for llm analog of moore’s law to get costs down.
In most cases, LLMs can get you 80-95% of the way, sometimes less, sometimes more. And heck, sometimes, it just gets you somewhere wrong.
But it seems everyone is arguing about whether LLMs can be perfect software engineers in isolation running in a closet, and using that to say that LLMs do not have a massive potential in other scenarios.
Sometimes, I like to imagine how much more productive most organizations could be from the things that the internet gave us, even to this day. Most companies never really do even a fraction of what is possible. That helps to ground my view of LLMs as well.
The fault dear Brutus isn't in our language models, but in ourselves.
Why on earth would we ever remotely compare a 'tool' to 'a software engineer' ?
The 'great delusion' is not that 'AI can't code' - because obviously it can, and very well.
The problem is the 'anthropomorphism' and all this AGI nonsense.
If we called it 'Stochastic Mechanisms' and did not 'personalize' our prompts, refer to them as 'chat' or give them 'personalities' but remained in the domain of 'Stochastic Language CLI' ... then our metaphors would pbably not cloud our judgments.
Let the philosophers argue about AGI.
Nailed it!
At my last place this was encouraged (by non-technical leadership driving the AI adoption policies, as well as setting salaries) and seen as a huge win.
The "step change in number of created PR's" was celebrated (cult-style), and by one of the (co) CEO's praised as a paradigm shift of the same magnitude as the personal computer. Meanwhile, I was stuck finding insta-reject level bugs in pull requests from people one-shotting 6000 line PR's "finally solving" long-standing issues from the backlog. Needless to say I left.
LeCun thinks that LLMs are a bad fit for AI that understands the physical, dynamical systems that we inhabit, and that understanding this is necessary for AGI/ASI.
I don't know that Hutter is bearish on LLMs, but Hutter is interested in AI that can reason exceptionally well given infinite compute, and approximations of such a reasoning AI. I think he is open to the idea that LLMs can be such an approximation.
being able to program is not the only skill required to be a successful software engineer, so no ai agents cannot be software engineers
very important distinction - i personally like the radiologist example - looking at scans is a part of a radiologist job, AI can do it better than most of them, but looking at scans is a small part of the job, most of it communicating with doctors to help their patients
I think the discussion about methods (coding agents included) depends on answering those questions. Seems pointless to claim these agents [dont] make you more productive.
Although, at a first glance, the productivity increase does seem like nothing I’ve seen before. Even more than the transition of making webapps in plain js -> jquery -> frameworks or going from something like Flask to using Rails.
Problem is this is not evidence based. I just feel prototyping has speed up 100x. So the number of iterations/attempts has gone up. Transforming specs into a test suite takes a fraction of the time. Dunno, feels weird not to be able to be overall more productive (do more with less time) if you have these new tools.
I can't agree with this. You tend to get one point of view, often without any actual resources and references so you have to go look it up yourself, on [insert search engine]. Plus, what does it say when we consider an AI the one stop for our data intakes.
This is dangerously incorrect. AI summaries of search results consistently return incorrect information and grossly oversimplified and thus misleading summaries, neither of which are detectable unless one either has prior domain knowledge or spends time drilling into search results to validate the AI output.
biosubterranean•1h ago