Isn't the sample super biased? StackOverflow is increasingly bleeding users to AI tooling. Shouldn't we expect the remaining users to be increasingly distrustful of AI?

I don't use StackOverflow at all anymore.

And they're getting more expensive

As expected we’re getting closer and closer to the Trough of Disillusionment. That’s not a bad thing, because it leads to the Plateau of Productivity.[1]

Anecdotally for myself I’m finding that LLMs are great when I can give it a hyper specific target like a function to write. This isn’t because it can’t write an entire script. It can. It’s because the more I let it run wild, it feels like my understanding of the code gets exponentially worse.

1: https://en.m.wikipedia.org/wiki/Gartner_hype_cycle

"AI tools are bad", says company most suffering by the use of AI tools

> https://survey.stackoverflow.co/2025/ai

One of the biggest problems I've run into AI tooling is they brute force a solution instead of subconsciously thinking about things that engineers subconsciously think about while crafting a solution, so even if it does manage to solve the problem you asked it solve it almost always isn't the best solution out there.

The reality of problems such as this one is you don't discover them until the product is in a user's hand for awhile, so while LLMs may be useful is speeding up the resolution of these problems they're kind of useless at discovering them. Yet despite this CEOs of companies with major AI investments are promising an idealized future within a couple of years as though it hasn't already been nearly 3 years of ChatGPT. Despite all the hype and promises the pace of the product development lifecycle seems to remain unchanged. I think realistically it'll be at least 10 years before LLM-based tools and agents can do what CEOs are promising.

After 2-3 weeks of going hard at Claude Max - yeah it's got major limits. At first I was trying to make full ready-to-go-to-prod entirely with vibe coding. Then I progressed to reviewing all the code it made. However, it will spend at least a third of it's tokens and my constantly ( 5 hr ) replenishing opus on spinning in a circle - checking random dirs, random files, and unrelated things for the answer to a problem that it THINKS is solution ( and that's not even considering the price increase ( by token reduction and 7 day waiting period lol ) that they will be doing in August.

Then I just stopped letting it run rampant and reviewed every step it took ( this is when I got the best results. ) So realistically - all it would be good at in a complex enterprise software stack ( My primary experience is with Guidewire ) it would be great for quickly scaffolding new parts of a micro service, adding onto pieces of one very specifically, and just removing the grunt work of manually hitting tab, cmd ->, ctrl ->, opt -> through big files and letting me just read it and confirm rather than getting carpal tunnel.

Honestly the best use I've gotten out of it has been updating and adding onto my emacs config primarily used for org-roam.

As far as replacing engineers? After at least 300 hours using max - I can say no it will not. I realized this after I spent more time configuring the rules and prompting the ai than actually just writing the code myself.