We've seen similar waves with new technologies before: overexposing the pros, dismissing the cons, hyper-optimism, and people using a lot of jargon without saying much of substance. The difference this time is the scale of the impact and the volume around it.
May I fix that for you?
The product is being launched before the value is there.
(I wouldn't suggest basing my evaluation of the entire field of coding agents around that particular product.)
That statement somehow amazes me. Obviously, LLMs need plenty of steering, but I don't get how a company can be built on that.
Again, are we still on this ridiculous concept. I stopped reading at that sentence.
Is that going to be the revisionist future that the industry tells itself - ""one day, LLMs came along and suddenly computers could program themselves, and it was an absolute revolution, but it made no difference at all to anything because "The hard part of software engineering was never writing the code." so people stopped using LLMs because of that. And that's how LLM's came to an end after being a short trend.""
The second part was that the test was being administered by a gatekeeper who was hired before those practices were put in place.
The 3rd was what? Amazon or Netflix burning your ass out because they are anti-human companies?
That Nvidia finances a lot of this should give anyone with a clue on finance pause, that it doesn’t tells you you’re in the middle of a bubble.
There are definitely guys in some companies just pointing an agent swarm at 50 Github tickets and saying "go get 'em boys" and hence need a lot of orchestration tools but for the rest of us do we really need the AI to do all the build steps and the testing steps... or do we just need the new feature that runs when the button is clicked and then we can check it out in the browser ourselves. The code gen is what I need the AI for, not the 'smoke tests' and the tsc finagling. I'm willing to spend some mental calories myself in figuring this feature out step by step
Actually this makes me wonder if some people are just not very into building step by step like "okay first let's check the JSON shape the endpoint returns", "okay now let's make a simple form", etc
I still don‘t know if this is real. If it works like that, where are the companies that only kept every tenth, 10x‘d AI enhanced dev? They should have significantly higher margins. The companies that keep all devs and 10x‘d them, finally bringing that backlog to 0 should have exploding revenue and profit.
Where does AI create value? I can‘t see it.
I use it every day, but nothing got faster.
Why fire anyone in this scenario? Growth is only going to keep coming.
This is one of the reasons that I've simply not bothered with a lot of these types of AI products. It feels like gambling. Maybe I'll spend $20 on tokens and end up with something awesome. Or maybe I'll spend $20 on tokens and end up with nothing useful and then I'll be glad it was only $20 I lost.
God this times 1000. I'm so sick of having AI chase a bug and having to lead it by the hand like a toddler to try and help me solve bugs. The difference is, a toddler learns when you explain. An LLM holds your responses in context and uses them to generate more authentic sounding garbage, but next session, foomf, the lesson is unlearned again.
I had a tiny problem with prerendering not working on a react site last night, I must have spent close to an hour running commands it asked me to run like a moron, verifying dumb things I had already checked like "did the file actually upload to ftp" or had I restarted Apache or was there some magic htaccess file somewhere. Eventually I just gave up on Claude and tried some different paths in Apache virtual host settings and it worked.
Of course if I went back to Claude and reported that, as would be my instinct with a human, it would confidently explain why that was the bug, why it couldn't spot the issue, and how smart I am for finding it. And then it would forget everything tomorrow
This hasn't been my experience at all. I can give it a vague description of the problem and have it find it pretty easily without any more input from me.
I do think there's a pretty big variation in tools and setups and what output people are getting. I'm now just using OpenAI Codex in VS Code and it churns through problems like they're nothing.
Sometimes I have to get it to not over-engineer a solution; not because it's necessarily wrong but because I don't need that much correctness.
I don't recommend web work, the last ten years or the next ten. Wouldn't touch the stuff. I skipped the phone app era, too.
It also wrote some unit tests that validated the kernel sampling weights, and wrote some Jupyter notebooks to go along with the kernel algorithms as comparisons.
It's not just web dev... It helps (a lot in some cases) if you ask very specific things rather than just "make this vague thing", but I'm more and more coming round to the conclusion it is now a useful dev tool (until two months ago I was a sceptic).
I do agree that the current pace of abandonware creation is wild. I think a lot of projects are generated top down from an idea, and not bottom up through usage. If you have a new experimental project, and you don't have recent commits, I assume you are not using it and thus it has no value.
I think agentic use is mostly only valuable as author extension (help you search, rubber duck, generate code but needs very heavy review). Over time with enough usage of a process I do think some of them can turn into author automation but not from simply writing down an idea. It takes lots and lots of executions, iterations, generalizations, specifications, basically lots of work to get any decently hands-off intelligent automation through agents.
No. Anyone making this claim is being disingenuous.
I still think of what I'm doing as software engineering, and I'm glad that I had many years of professional and hobby development before using agents since I think that's given me the ability to make good architectural decisions (and helps me resteer the LLMs when they want to do something suboptimal), but my involvement in actually writing code is quickly going to zero.
That said, they aren't perfect and they still introduce bugs, but I believe the quality of my current product is higher than what I would have created pre-agentic coding.
Things I've found helpful in keeping quality high: - Visual regression tests (detect UI bugs before you commit them) - Fuzz testing of interfaces and app behavior - Automatically add regression tests for any bug that I/the LLM fixes - Logging/alerting that tracks an errors/invariant violations triggered in the app - Performance metrics that are surfaced in a dashboard.
All of these are very easy to add since the LLM can create this infrastructure for you. The fuzz testing in particular is something very few products I've previously worked on have since most people don't know how to implement it. I ran the fuzzers for a few minutes and they quickly caught multiple subtle bugs that I was not aware of.
This is a real product that helps a real, non-VC funded service business, and although I could have made something similar myself it would have taken me a lot longer, be harder to use, and probably be less reliable.
Edit: while it's true that you can quickly blow through the $20/month plan, the $200/month plan allows you to get a lot done and is basically sufficient for my needs. It's also very cheap when you consider what it would cost to pay someone to do similar work.
Two years ago, I couldn't just roughly describe my backlog and then have the code fixed. I had to type it out myself, run it, look at logs, fix toolchain issues, and so on. It was tedious. Or I could get a junior to do it.
Now can get these things done quite fast, without concentrating nearly as hard.
Clearly, it isn't vapour.
It delivers something. That something we have yet to figure out the best way to use, but there's definitely something there that works.
I get the feeling a lot of people are frustrated because the little gains are lost in organisational chaos, rather than the tools not working.
I think its fascinating just how much of a gap there is between what's being claimed, and the verifiable observable data of the open source world. Major open source projects are by and large starting to ban LLMs now, because the contributions made by LLM users have been universally terrible and unhelpful. There doesn't appear to be a single major project that's found generating code to lead to major productivity speedups, and the consensus appears to be that its just lead to a lot of crappy contributions that are harder to spot immediately as being obvious crap
I regularly see people claim that they are now 10x more productive with LLM code generation, and I just wonder where all the code is. Is it somehow true that these gains are only being realised in proprietary projects, and not a single one of them has put even a small fraction of their new found engineering powers into eg Godot? Why do only the poor quality LLM code generation users make PRs to open source projects, and never the engineers that know how to really use it correctly?
If you look in the open source major project space, you can find almost no evidence that AI code generation exists at all. Go browse your favourite critical tool and look for AI generated PRs that have landed in the codebase, its probably a tiny handful of them in comparison to the human written PRs prior to an LLM ban. It turns out that once you have a verifiable, open quality review bar, for some reason almost no LLM commits really meet the level of quality necessary
I strongly suspect that what we're seeing is that much of the tech code-writing economy had already become completely performative prior to AI turning up. It no longer matters in the current age if your code is good, or works, because your job is to give the illusion of product development while the stock market price gets pumped, until you all cash out your share value, get bought, or hop jobs in 2 years. For many companies it literally does not matter if you produce anything that generates value (or works), because the illusion of progress is all that matters. AI is absolutely incredible at creating the illusion of progress, because it looks a whole lot like real code, it just appears to have failed the bar of making actual projects that work. If that was never the goal in the first place, it probably really is a 10x productivity boost
I almost wonder if there's a campaign to start suppressing public awareness of where SOTA capabilities really are.
That's like saying smart watches are useless based on trying out the smart watch made by will.i.am's tech company rather than Apple's.
There's no such thing as a free lunch, and anyone trying to build software without significant guardrails and insights into the process will have to give up control of their codebase.
Ray Myers does a good job of exploring this topic in a recent Software Should Work talk, trying to answer: "Is code for people or AI?" [0]
I don't think devs are that expensive relative to a big company.
Airbnbs revenue last year was 12b. If they have 1000 engineers,maybe average 200k/yr that's only 200m. If they got rid of their engineers still only a fraction of of the revenue
Then it probably would have taken me at least half a day to think about what tests to write and to make them.
It did it in around 5 minutes. I had to check it, and it wasn't perfect, but it compiled, ran, and was very close.
Claude Opus 5 is insane. Still not able to do big complex projects well start to finish but it can, with guidance, implement large parts of them.
There are now open weights models approaching it in performance too.
People keep repeating your sentiment here but I simply can't follow, are we even on the same planet? Or did everyone switch to just not caring about maintainability and code quality anymore? Or are your work tasks simply so mindnumbingly, stupidly simple that even an AI can oneshot them properly?
I mean this honestly btw, not dismissively like some sister comments. The gap between the productivity increases people report on HN and what I experience myself is insane. In fact, if I factor in the procrastination I find myself doing on dotting the i's on a supposedly "one-shotted" AI implementation of a nontrivial feature, I think the AI actively slows me down.
The only way I've found that I can actually use AI productively and sustainably is in very small tight loops and, well, at that point it's not that much faster than just typing in the code (with the occasional "Cursor Tab" complete).
Am I doing something wrong?
If AI is as good as it being marketed as, you shouldn't even be able to hold it wrong!
A big problem is that these codebases rot. Agents move incredibly fast at first, but then as you pay less attention (or perhaps no attention at all) to the architecture, they slowly fall to bits. So then you decide, I'll use AI to rewrite it! And it gets better for a while until, well, you get it.
That's not to say there isn't value here, there absolutely is, just — chill. A little.
LLMs will certainly be an aide, but assuming adoption of them is required across a whole _field_ ignores whole classes of problems, domains, and constraints the production of software covers.
Rather, we should be wary of allowing our skills and hard-earned knowledge to atrophy by over reliance on new technology that is far from perfect, reliable, or even universally available. These tools more than any before suffer from the junk-in-junk-out problem and I’d much rather work with someone who knows their fundamentals than someone who knows their way around a the LLM of the month.
Edit: although you might be subject to an NDA... But this is pretty much my test for "AIs will take all the jobs": can it write truly safety-critical software yet?
I suspect the ban of AI in established projects to be a very complex decision. Even if core developers would like to use AI, they don't want to review all of the AI-generated code from the larger community. So the only consistent way to preserve sanity is to declare that nobody would use it. Good luck enforcing it though!
Nothing's stopping core developers from adopting LLM generated code for themselves, while banning it for external contributors
So 0<whatever<1
It could be that the engineers who are extremely productive with LLMs are landing PRs that look indistinguishable from good, hand-written PRs.
LLMs are superhuman at short term coding such as debugging and writing tests though and you're missing out by not using them there.
Instead, software is plodding along exactly the same as it did prior to LLM code generation, and there's no evidence of superprogrammers making superprojects in 1/10th of the time. With a 10x productivity gain, what used to take a year should take a month
It seems likely that the gains from generating tons of code are likely being offset by the debt incurred to understanding what you're doing. We see a ton of new greenfield projects one-shotted with GPT or GLM or whatnot, but very little on the side of projects with long-term maintenance goals. This is telling, to me.
It's not just productivity, it's life changing.
EA-3167•49m ago
cyanydeez•44m ago
I can absolutely see where this technology works; but like EVERYTHING THAT THESE VC FUNDED FASCISTS IGNORE, just because it works in the lab does not mean it works in real life. I can see exactly where this technology fails: it lets you spool out so much thread into the abyss that you're just going to keep refactoring everything, all the time and when it blows a hole in your project, whelllp! you wanted to refactor that too. Imagine having 100 side projects grossly built and still, you just want to move on.
Fiishing projects are still going to need work and the explosion of "this is _my_ engineering harness" is everyone twiddling their thumbs cause real projects of significant value are still about scope and "total addressable market" and not "I made an AI to diddle children, so you dont have to" crowd.