(I'm not really offended honestly. Startups will come crying to un-vibe the codebases soon enough.)
This is confusing.. it's directly saying AI is improving employee productivity, but that's not leading to more business profit... how does that happen?
One trivial way is that the increase of productivity is less than the added cost of the tools. Which suggests that (either due to their own pricing, or just mis-judgement) the AI companies are mis-pricing their tools. If the tool adds $5000 in productivity, it should be priced at $4999, eventually -- the AI companies have every motivation to capture nearly all of the value, but they need to leave something, even if just a penny, for the purchasing company to motivate adoption. If they're pricing at $5001, there's no motivation to use the tool at all; but of course at $4998 they're leaving money on the table. There's no stable equilibrium here where the purchasing companies end up with a /significant/ increase in (productivity - cost of that productivity), of course.
Sounds like the AI companies are not so much mispricing, as the companies using the tools are simply paying wayyy too much for the privilege.
As long as the companies keep paying, the AI companies are gonna keep the usage charges as high as possible. (Or at least, at a level as profitable to themselves as possible.) It's unreasonable to expect AI companies to unilaterally lower their prices.
For some reason, I'm thinking most of the money went to either inferencing costs or NVidia.
Executives mistook that novelty for a business revolution. After years of degraded search, SEO spam, and “zero-click” answers, suddenly ChatGPT spat out a coherent paragraph and everyone thought: my god, the future is here. No - you just got a glimpse of 2009 Google with autocomplete.
So billions were lit on fire chasing “the sliced bread moment” of finally finding information again - except this time it’s wrapped in stochastic parroting, hallucinations, and a SaaS subscription. The real irony is that most of these AI pilots aren’t “failing to deliver ROI” - they’re faithfully mirroring the mediocrity of the organisations deploying them. Brittle workflows meet brittle models, and everyone acts surprised.
The pitch was always upside-down. These things don’t think, don’t learn, don’t adapt. They remix. At best they’re productivity duct tape for bored middle managers. At worst they’re a trillion-dollar hallucination engine being sold as “strategy.”
The MIT study basically confirms what was obvious: if you expect parrots to run your company, you get birdshite for returns.
In this case, that's NVDA
Crypto's over, gaming isn't a large enough market to fill the hole, the only customers that could fill the demand would be military projects. Considering the arms race with China, and the many military applications of AI, that seems the most likely to me. That's not a pleasant thought, of course.
The alternative is a massive crash of the stock price, and considering the fact that NVIDIA makes up 8% of everyone's favorite index, that's not a very pleasant alternative either.
It seems to me that an ultra-financialized economy has trouble with controlled deceleration, once the hypetrain is on it's full-throttle until you hit a wall.
Data centers might, but then they'll need something else to compute, and if AI fails to deliver on the big disruptive promises it seems unlikely that other technologies will fill those shoes.
I'm just saying that something big will have to change, either Nvidias story or share price. And the story is most likely to pivot to military applications.
- everyone and their mother are doing a "generative ai program" right now, a lot of times just using the label to try to get their project funded, ai being an afterthought
- if the 1 out of 20 projects is game-changing, then you could argue right now people should actually be willing to spend even more on the opportunity, maybe the number should actually be 1 in 100. (The VC model is about having big success 1 in 10 times.)
- studies of ongoing business activities are inherently methodologically limited by the data available; I don't have a ton of confidence that these researchers' numbers are authoritative -- it's inherently impossible to truly report on internal R&D spend especially a private companies without inside information, and if you have the inside information you likely don't have the full picture.
It’s all fun and games until the bean counters start asking for evidence of return on investment. GenAI folks better buckle up. Bumps ahead. The smart folks are already quietly preparing for a shift to ride the next hype wave up while others ride this train to the trough’s bottom.
Cue a bunch of increasingly desperate puff PR trying to show this stuff returns value.
"Hey, guys, listen, I know that this just completely torched decades of best practices in your field, but if you can't show me progress in a fiscal year, I have to turn it down." - some MBA somewhere, probably, trying and failing yet again to rub his two brain cells together for the first time since high school.
Just agentic coding is a huge change. Like a years-to-grasp change, and the very nature of the changes that need to be made keep changing.
Agents may be good (I haven't seen it yet, maybe it's a skill issue but I'm not spending hundreds of dollars to find out and my company seems reluctant to spend thousands to find out) but they are definitely, definitely not general superintelligence like SamA has been promising
at all
really is sinking in
these might be useful tools, yes, but the market was sold science fiction. We have a useful supercharged autocomplete sold as goddamn positronic brains. The commentariat here perhaps understood that (definitely not everyone) but it's no surprise that there's a correction now that GPT-5 isn't literally smarter than 95% of the population when that's how it was being marketed
You really set yourself up with a nice glass house trying to make fun of the money guys when you are essentially just moving your own goal posts. It was annoying two (or three?) years ago when we were all talking about replacing doctors and lawyers, now it just cant help but feel like a parody of itself in some small way.
"Donald Trump and Silicon Valley's Billionaire Elegy" - https://www.wired.com/story/donald-trump-and-silicon-valleys...
That said, technologies like this can also go through a rollercoaster pattern itself. Lots of innovation and improvement, followed by very little improvement but lots of research, which then explodes more improvements.
I think LLMs have a better chance at following that pattern than computer vision did when that hype cycle was all the rage
"Spending on AI data centers is so massive that it’s taken a bigger chunk of GDP growth than shopping" - https://fortune.com/2025/08/06/data-center-artificial-intell...
Here's the source report, not linked to by this content farm's AI-written article: https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Bus...
Trying to claim victory against AI/US Companies this early is a dangerous move.
Too young to remember GSM?
[0]https://www.researchgate.net/figure/Napoleon-march-graphic-C...
1. Generate content to create online influence. This is at this point probably way oversaturated and I think more sophisticated models will not make it better.
2. Replace junior developers with Claude Code or similar. Only sort of works. After all, you can only babysit one of these at a time no matter how senior you are so realistically it will make you, what, 50% more productive?
3. Replace your customer service staff. This may work in the long run but it saves money instead of making money so its impact has a hard ceiling (of spending just the cost of electricity).
4. Assistive tools. Someone to do basic analysis, double check your writing to make it better, generate secondary graphic assets. Can save a bit of money but can’t really make you a ton because you are still the limiting factor.
Aside: I have tried it for editing writing and it works pretty well but only if I have it do minimal actual writing. The more words it adds, the worse the essay. Having it point out awkward phrasing and finding missing parts of a theme is genuinely helpful.
5. AI for characters in video games, robot dogs, etc. Could be a brave new frontier for video games that don’t have such a rigid cause/effect quest based system.
6. AI girlfriends and boyfriends and other NSFW content. Probably a good money maker for a decade or so before authentic human connections swing back as a priority over anxiety over speaking to humans.
What use cases am I missing?
As for relying on the code base, that is good for code, not for onboarding/deployment/operations/monitoring/troubleshooting that have manual steps.
We connect with slack/notion/code/etc so that you can do the following:
1. Ask questions about how your code/product works 2. Generate release notes instantly 3. Auto update your documentation when your code changes
We primarily rely on the codebase since it is never out of date
How much does that cost these days? Do you still have to fly to remote islands?
Sorry this is some bull. Either it works or it doesn’t.
How many hundreds of hours is your team spending to get there? What is the ROI on this vs investing that money elsewhere?
It is uniquely susceptible because the gaming market is well acclimated to mediocre writing and one dimensional character development that’s tacked on to a software product, so the improvements of making “thinking” improvisational characters can be immense.
Another revenue potential you’ve missed is visual effects, where AI tools allow what were previously labor intensive and expensive projects to be completed in much less time and with less, but not no, human input per frame
I mostly disagree. Every gaming AI character demo I've seen so far is just adds more irrelevant filler dialogue between the player and the game they want to play. It's the same problem that some of the older RPG games had, thinking that 4 paragraphs of text is always better than 1.
But if you're actually trying to provide good customer service because people are paying you for it any paying per case then you wouldn't dare put a phone menu or AI chat bot in-between them and the human. The person handles all the interaction with the client and then uses AI where it's useful to speed up the actual work.
I don't know why everyone goes to "replacing". Were a bunch of computer programmers replaced when compilers came out that made writing machine code a lot easier? Of course not, they were more productive and accomplished a lot more, which made them more valuable, not less.
Like is the conclusion we shouldn't even try? This kind of thinking ridiculous.
What menial about knowledge work, anyway?
I do want people to understand what they are discarding when they use it to replace human creativity and human contact. All the evidence is that AI bros still think artists are evil gatekeepers.
And I absolutely want to see the bubble burst. I see absolutely no reason to be excited on the behalf of Silicon Valley vampires and their latest, soul-crushing innovation. If it’s making Andreessen Horowitz a stack of money, its value to humanity is already more questionable.
While people are doing their work, they don't think, "Oh man, I am really excited to talk with AI today, and I can't wait to talk with a chatbot."
People want to do their jobs without being too bored and overwhelmed, and that's where AI comes in. But of course, we cannot hype features; we sell products after all, so that's the state we are in.
If you go to Notion, Slack, or Airtable, the headline emphasizes AI first instead of "Text Editor, Corporate Chat etc".
The problem is that AI is not "the thing", it is the "tool that gets you to the thing".
In reality, AI sparkles and logos and autocompletes are everywhere. It's distracting. It makes itself the star of the show instead of being a backup dancer to my work. It could very well have some useful applications, but that's for users to decide and adapt to their particular needs. The ham-fisted approach of shoving it into every UI front-and-center signals a gross sense of desperation, neediness, and entitlement. These companies need to learn how to STFU sometimes.
Too many companies are just trying to spoon AI into their product somehow, as if AI itself is a desired feature, and are forgetting to find an actual user problem for it to actually solve.
I could be wrong but, all in all, buy a .com for your "ai" product, such that you survive the Dot-ai bubble [1]
For example a study from METR found that developers felt that AI sped them up by 20%, but it empirically it slowed them down by 19%. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...
How you use AI will depend on the model, the tools (claude-code vs cursor vs w/e), your familiarity and process (planning phases, vibe coding, etc.), and the team size (solo dev versus large team), your seniority and attention to detail, and hard to measure effects like an increased willingness to tackle harder problems you may have procrastinated on otherwise.
I suspect we're heading to a plateau. I think there's a ton of polish that can be done with existing models to improve the coding experience and interface. I think that we're being massively subsidized by investors racing to own this market, but by the time they can't afford to subsidize it anymore, it'll be such a commodity that the prices won't go up and might even go down regardless of their individual losses.
As someone who knows they are benefitting from AI (study shmuddy), I'm perfectly fine with things slowing down since it's already quite good and stands to be much better with a focus on polish and incremental improvements. I wouldn't invest in these AI companies though!
They got a majority of the country hooked into AI without truly understanding its current limitations. This is just like digital currency bubble/fad that popped a couple of years ago.
What most companies got out of it is a glorified chatbot (ie, something that was possible in 2014…) at 1000X the cost.
What a sad joke. Innovation in this country is based on a lie, fueled by FOMO.
Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually.
We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls.
We recently switched to using AI to transcribe and write these summaries. Not only are the summaries better than those produced by our human agents, they also free up the human agents to do higher-value work.
It's not sexy. It's not going to replace anyone's job. But it's a huge, measurable efficiency gain.
Of course, we can just rely on knowing nothing just to look things up, but I want more for thinking peoples.
I'm finding that the summarization of individual meetings very useful, I'm also finding that the ability to send in transcripts across meetings, departments, initiatives whatever to be very effective at surfacing subtexts and common pain points much more effectively than I can.
I'm also using it to look at my own participation in meetings to help me see how I interact with others a (little) bit more objectively and it has helped me find ways to improve. (I don't take its advice directly lol, just think about observations and determine myself if it's something that's important and worth thinking about)
We employ a loop where we generate the PPTX, it goes to the team for polish, they submit the final PPTX back, and the system assesses the differences between provided 'observations' from the LLMs and what the humans eventually delivered.
In 90% of cases, during our regular human review sessions, the LLMs (after much tweaking of context and prompts to make them sound less...flowery) are providing subjectively better insights than many of the humans. There are a wide variety of reasons for this, but it's primarily available time to deliver (too much to do; too wide of a range of slides to cover; etc, etc, etc). If the slide insights are just 'reporting the weather', the LLM can do that just fine. However, where the LLMs struggle, but humans outpace them, is in the real meaty insights that cross into areas where context about the business or the overall efforts are unavailable to the LLM.
In our case, we have the subjectively 'best' SMEs and insight writers evaluating the insights cross-client deliverables on a regular cadence and helping us tweak not only the LLMs but better training the humans on how to write better insights. Assuming there is a human eval in the loop, you're probably going to get a better system in place over time.
What I see in businesses trying to leverage LLMs is that they just assume they can replace everything w/the LLM and move along w/o supervision. This is nonsense and that sort of assumption, while likely driven by the AI-marketing-hype-cycle, is just batshit crazy to me. Due diligence is required at every step and it will pay off, just like any technology, but we want to pretend it isn't.
Is there some training you applied or something specific to your use case that makes it work for you?
When was the last time you called a large company and the person answering was already across all the past history without you giving them a specific case number?
Finally, who cares about millions saved (while considering the above introduced risk), when trillions are on the line?
It's a tad far-fetched in this specific scenario, but an AI summary that says something like "cancel the subscription for user xyz" and then someone else takes action on that, and XYZ is the wrong ID, what happens?
AI today is terrible at replacing humans, but OK at enhancing them.
Everyone who gets that is going to find gains - real gains, and fast - and everyone who doesn't, is going to end up spending a lot of money getting into an almost irreversible mistake.
Now, summary, or original? (Provided the summary is intentionally vague to a fault, for arguments sake on my end).
Is it not, in the scenario you are describing? You are saying the agents are free now to do higher-value work. Why were there not enough agents before, especially if higher-value work was not done?
its likely a checkbox for compliance or some policy a middle manager put in place that is now tied to a kpi
With LLMs the risk is particularly hard to characterize, especially when it comes to adversarial inputs.
There, I've saved you more millions.
Edit: Tell me more how preemptively spending five figures to transcribe and summarize calls in case you might want to do some "data engineering" on it later is a sound business decision. What if the model is cheaper down the road?
But in fact, customer call centers tend not to be able to even know that you called in yesterday, three days ago and last week.
This is why email-ticketing call centers are vastly superior.
I'm not going to say every project born out of that data makes good business sense (big enough companies have fluff everywhere), but ime anyway, projects grounded to that kind of data are typically some of the most straight-forward to concretely tie to a dollar value outcome.
Why OPUS though? There's dedicated audio codecs in the VoiP/telecom industry that are specifically designed for the best size/quality for voice call encoding.
> It's not going to replace anyone's job
Mechanically, more efficiency means less people required for the same output.
I understand there is no evidence that any other sentence can be written about jobs. Still, you should put more text in between those two sentences. Reading them so close together creates audible dissonance.
Why were they doing this at all? It may not be what is happening in this specific case but a lot of the AI business cases I've seen are good automations of useless things. Which makes sense because if you're automating a report that no one reads the quality of the output is not a problem and it doesn't matter if the AI gets things wrong.
In operations optimization there's a saying to not go about automating waste, cut it out instead. A lot of AI I suspect is being used to paper over wasteful organization of labor. Which is fine if it turns out we just aren't able to do those optimizations anyway.
It was equally frustrating when I, as a call center worker, had to ask the custmer to tell me what should already have been noted. This has required me to apologize and to do someone else's work in addition to my own.
Summarizing calls is not a waste, it's just good business.
It's 100% plausible it's busy work but it could also be for: - Categorizing calls into broad buckets to see which issues are trending - Sentiment analysis - Identifying surges of some novel/unique issue - Categorizing calls across vendors and doing sentiment analysis that way (looking for upticks in problem calls related to specific TSPs or whatever) - etc
False positives and negatives aren't really a problem once you hit a certain scale because you're just looking for trends. If you find one, you go spot-check it and do a deeper dive to get better accuracy.
Which is also how you end up with some schlepp like me listening to a few hundreds calls in a day at 8x speed (back when I was a QA data analyst) to verify the bucketing. And when I was doing it everything was based on phonetic indexing, which I can't imagine touching llms in terms of accuracy, and it still provided a ton of business value at scale.
However I strongly doubt your point about "It's not going to replace anyone's job" and that "they also free up the human agents to do higher-value work". The reality in most places is that fewer agents are now needed to do the same work as before, so some downsizing will likely occur. Even if they are able to switch to higher-value work, some amount of work is being displaced somewhere in the chain.
And to be clear I'm not saying this is bad at all, I'm just surprised to see so many deluded by the "it won't replace jobs" take.
Imagine a human agent or AI summarises: “Customer accepted proposed solution.” Did they? Or did they say “I’ll think about it”? Those aren’t the same thing, but in the dashboard they look identical. Summaries can erase nuance, hedge words, emotional tone, or the fact the customer hung up furious.
If you’re running a call centre, the question is: are you using this text to drive decisions, or is it just paperwork to make management feel like something is documented? Because “we saved millions on producing inaccurate metadata nobody really needs” isn’t quite the slam dunk it sounds like.
This reminds me of the way juniors tend to think about things. That is, writing code is "the actual job" and commit messages, documentation, project tracking, code review, etc. are tedious chores that get in the way. Of course, there is no end to the complaints of legacy code bases not having any of those things and being difficult to work with.
They are hilariously inaccurate. They confuse who said what. They often invert the meaning "Joe said we should go with approach x" where Joe actually said we should not do X. It also lacks context causing it to "mishear" all of our internal jargon to "shit my iPhone said" levels.
But that doesn’t mean AI is without its uses. We’re just in that painful phase where the hype needs to die down and we treat LLMs as what they really are; an interesting new tool in the toolkit that provides some new ways to solve problems. It’s almost certainly not going to turn into AGI any time soon. It’s not worth trillions. It’s certainly worth something, though.
I think the financials on developing new frontier models are terrible. But I’ve already built multiple AI projects for my company that are making money and we’ve got extremely happy customers.
Investors thought one company was going to win the AI Wars and make a quadrillion dollars. Instead it’s probably going to be 10,000 startups that will build interesting products based on AI, and training new models won’t actually be a good financial move.
Did users knew that conversation was recorded?
We have someone using Firefly for note taking, and it's pretty bad. Frequently gets details wrong or extrapolates way too much from a one-off sentence someone said.
How do you verify these are actually better?
This is a tiny fraction of all work done. This is work people were claiming to have solved 15 years ago. Who cares?
It's also disappointing that MIT requires you to fill out a form (and wait for) access to the report. I read four separate stories based on the report, and they all provide a different perspective.
Here's the original pdf before MIT started gating it: https://web.archive.org/web/20250818145714/https://nanda.med...
Specifically: Do they spend more time actually taking calls now? I guess as long as you're not at the burnout point with utilization it's probably fine, but when I was still supporting call centers I can't count the number of projects I saw trying to push utilization up not realizing how real burnout is at call centers.
I assume that's not news to you, of course. At a certain utilization threshold we'd always start to see AHTs creep up as agents got burned out and consciously or not started trying to stay on good calls.
Guess it also partly depends on if you're in more of a cust serv call center or sales.
I hated working as an actual agent on the phones, but call center ops and strategy at scale has always been fascinating.
Incorrect. He did check, and decided to lie.
... Well, probably yes, but I don't have the data to do it.
From the article though:
> But researchers found most use cases were limited to boosting individual productivity rather than improving a company’s overall profits.
What does that even mean?
[1] Website: https://nanda.media.mit.edu/, FAQ: https://projnanda.github.io/projnanda/#/faq_nanda
I honestly don't think it matters though. Feel free to disagree with me but I think the money is irrelevant.
The only thing that actually matters is the long run is the attention, time, and brain space of other people. After all that's where fiat currency actually derives it's value. These Gen AI companies have captured a lot of that extremely quickly.
OpenAI might have "burned" billions but they way they have wrung themselves into seemingly every university student's computer, every CEOs mind, the policy decisions of world leaders, ever other hackernews post, is nothing short of miraculous...
Saved them hours of work.
Of course, they didn't spend on "AI" per se.
Most people don't know how to meta their job functions, so AI won't really be worth it. And the productivity gains may not be measurable ie: "I did this in 5 minutes instead of 500, so I was able to goof off more."
I think the real problem is, it's just a bit too early, but every CEO out there dreams of being lauded for their visionary take on AI, and nobody wants to miss the bus. It's high-leverage tech, so if it (some day) does what it's supposed to do, and you miss making the investment at the right time, you're done.
If you do not do it, you get left behind and cannot compete in the marketplace.
I took a business systems administration course like 20 years ago, and they knew this was the case. As far as we can tell it's always been the case.
IT doesn't create massive moats/margins because price competition erodes the gap. And yet if you do not keep up you lose.
It's definitely a boon for humanity though, in the industries where technology applies things have been very obviously getting much cheaper over time.
(Most notably American housing has been very very resistant to technological change and productivity gains, a part of the story why housing has gone way up) - https://youtu.be/VfYp9qkUnt4?si=D-Jpmojtn7zV5E8T
I want to know more about the 5% who got it right. What are their use cases ?
The article does call out clear issues companies have with AI workflows etc. and those are likely real problems, but if you're saying *zero* return those aren't the root cause problems.
AI is predominantly replacing outsourced, offshore workers
https://news.ycombinator.com/item?id=44940944
PDF report that was taken down/walled: https://web.archive.org/web/20250818145714/https://nanda.med...
It 100% turned out to be a bubble and yet, if anything, the internet was under-hyped. The problem in 1999 was that no one really knew how it was going to play out. Which investments would be shrewd in retrospect, and which ones would be a money pit?
When an innovation hits, it takes time to figure out whether you're selling buggy whips, or employing drivers who can drive any vehicle.
Plenty of companies sunk way too much money into custom websites back in 99, but would we say they were wrong to do it? They may have overspent at a time when a website couldn't justify the ROI within 12 months, but how could they know? A few short years later, a website was virtually required for every business.
So are companies really seeing "zero return" on their AI spend, or are they paying for valuable lessons about how AI applies to their businesses? There may be zero ROI today, but all you need to do is look at the behavior of normal people to see that AI is not going anywhere. Smart companies are experimenting.
So their feature is not just text to speech, but a reading of a summarized version of the articles. But here is the problem. The documentation has no fluff. You don't want a summary, you want the actual details. When you are reading the document that describes how the recovery fee is calculated, you want to know exactly how it is calculated.
I've ran it on multiple documents and it misses key information. An unsuspecting user might take it at face value. So this feature looks impressive, but it misses the entire point of documentation. Which is *preserving the details*.
It’s pretty clear to anyone who’s using this technology that it’s significant. Theres still tons to work out and the exact impact is still unknown. But this cat isn’t going back in the bag.
The story there is very different than what's in the article.
Some infos:
- 50% of the budgets (the one that fails) went to marketing and sales
- the authors still see that AI would offer automation equaling $2.3 trillion in labor value affecting 39 million positions
- top barriers for failure is Unwillingness to adopt new tools, Lack of executive sponsorship
Lots of people here are jumping to conclusions. AI does not work. I don't think that's what the report says.
Well...
"It is difficult to get a man to understand something when his salary depends upon his not understanding it"
rogerkirkness•1h ago
lazide•1h ago