Same, even opus favor short term solution and scripts with a billion flags that constabtly require rescanning to understand how to launch it is a constant struggle to get it to build sane default and reusable scripts that run with minimal parameters

Agreed. I’ve been telling my team to build up internal packages so we can push all that ad hoc reinvention into something more tangible and deterministic. Invest the $$$ in inference into something the agent can reach for next time that’s neutral and consumable by other code to reduce future spend.

AI can do things around semantic analysis that a deterministic diff tool cannot.

I understand and agree with your point though.

it's all about cost at the end of the day. if you're allowed and encouraged to tokenmaxx, then of course this'll happen.

I agree, but even this use case isn't the most wasteful. The interwebs says Agentic consumes 50% of token use, but I'd hazard this number is north of 90% for many shops. My cynical view of Agentic is its sole purpose is to make "number go up".

I have exposure to AI initiatives at several companies including a few F500's. I have seen teams dump huge logs into frontier models that took hours to get so-so results that we were able to replace with a few lines of python code at 1000 times the speed and 100% accuracy. When asked why they were doing this they literally said "because we don't understand the subject matter so we were depending on the AI". I saw one team file a complaint with a vendor about a frontier backed coding harness and it's inability to consistently format headers because they were using it as a reporting engine. When I recommended they just use the coding tool to write code to generate reports you would have thought I had just cured cancer from their response. I frequently see people complain about the fact that AI is going to take their jobs and then see them gripe about the fact that AI is 'worthless' because it can't do more of their job than it already does. It's easy to see the difference between the people seeing 10x productivity gains from leveraging AI and those who aren't and it's not the AI.

Oh no! People are doing what they've been told to do!

Laziness, pure and simple. The inevitable consequence of “the LLm is the compiler now”. And what do you even expect people to do when they are forced at threat of termination to use AI for everything as much as possible? Not to mention people are being pressured to do insane thing like review hundreds of pull requests per day and deliver like 15 features per week so OBVIOUSLY there isn’t time to build out proper tooling. Just shove everything in a prompt and call it a day. Some people have families to feed, just do what you’re told.

It's this and worse. To use your example, it's like people using AI to write a diff algorithm, incorrectly, then using AI to fix it, because they don't know that diff exists already. Lazyness and starting development with a very low level of understanding. People think lowering the barrier to entry is a good thing, when in reality there are just fundamentals and things you just have to know before you can start using a tool like llms properly.

Same with writing boilerplate code. It’s been a solved problem yet here we are.

Isn't that the supposed point of it though? At least how it is marketed/hyped. Don't use your brain, you don't need one, spend all your thinking energy on... dunno, something else, and leave all the "mundane" stuff to AI. Just pay for the tokens, it's going to make you 10x more efficient, the $1000/month is worth it.

Because you look at the work from the perspective of a programmer, not the perspective of a regular person.

Normal people have never gone around automating their work. The most automation they do is dynamic tables on excel sheets.

I obviously know building a tool that can programmatically do something is a better solution, but I think that requires a fundamental shift in how people work. People need to be told by someone "this is how you should be using the AI" but right now they're simple told "use the AI".

100% this. For my own company I mostly build deterministic workflows that may have a simple AI step in the middle using an appropriate Chinese model in a very limited way. I wouldn't want to burn tokens to satisfy some metric.

With this AI is a fallback and not the default. Sounds like large companies have it backwards.

Productivity gains seem like it’s at best a wash when you factor in the massive tech debt cleanup and additional time needed to spec and review.

> I have yet to see an end user product that in itself isnt a wrapper around LLMs that is impressive created by LLM assistance.

I don’t disagree that AI is overhyped. But I think you are probably looking in the wrong place.

I think most software that is written isn’t really a product, at least not a public product. It’s an in-house tool or a one-off project needed to complete some larger task. People everywhere are always writing small programs that make their life or job just a bit easier (and explains why so many corporate projects are little more than an excel spreadsheet).

And there are a lot of people who have made custom software just for themselves with AI. Not a product, just a tool or project that finally made sense to build.

elaborate please, how does it not work?

Yesterday I updated our dependency on the sqlx crate and put up a PR, and it failed in the CI build in a way I couldn’t reproduce locally.

I asked codex to take a look, and it:

- Grabbed the CI logs on its own to figure out what the CI error was

- Looked at my local setup

- Looked at the changes in sqlx from 0.8 to 0.9

And figured out that sqlx depends on an updated version of the “whoami” crate but doesn’t specify default features, which causes it to fall back on a stub implementation that makes the default user “anonymous”, which was failing to authenticate to the UNIX socket we use in our CI Postgres server. It patched the environment variable for our docker container to explicitly specify a username and the issue was fixed.

It would’ve taken me probably several hours to figure this out on my own. It took codex maybe 5 minutes.

Tell me again how LLM’s “don’t work”?

The actual cost is going to drop 99% in ~4 years.

How much that makes it into enterprise pricing is TBD, since none of the hyper scalers are making money yet of selling AI inference.

Almost all businesses are ahead of the gun. For most of their use cases, AI is either not yet good enough on its own, or good enough but too expensive.

No one wants to get left behind, so everyone's trying to get onto it now, even though it's not ready for what most enterprises want to do with it.

It's easy for them to look at a small startup without billions of lines of legacy business logic debt and see them having success and wonder why they can't have just as much - or more - why they're bigger so they should have better and more success, right???

Wrong...

But when it gets ~99% cheaper for local inference over the next 4 years, at the same time the price per watt improve 4x -> a lot of those cases will start to pencil out.