I don’t think they got it right and the market share and usage grew faster than inference dropped, but inference costs will clearly drop and these companies will eventually be very profitable.
Reality is that startups like this assume moore’s law will drop the cost over time and arrange their business around where they expect costs to be and not where costs currently are.
It could also be that you give too much credit to the market. People follow trends because in most cases that makes money. There is no other deeper though involved. Look at the financial crisis, totally irrational.
Inference costs for old models will drop, but inference costs may stay the same if models continue to improve.
No guarantee that any wrapper for inference will be able to hold on to customers when they stop selling $1.00 for $0.50.
Thanks!
Price-performance though? The trend is clear: a given level of LLM capability keeps getting cheaper, and that trend is expected to hold. Improvements in architecture and training make LLMs more capability-dense, and advanced techniques make inference cheaper.
One of the main selling points to MOE, is that the architecture is designed such that you can re-train experts independently, as well as add new experts, change the size of an experts parameters, etc, without retraining the entire model.
If 80% of you usage comes from 20% of your experts, you can cut your future training costs SUBSTANTIALLY.
It's not entirely impossible, but I remain skeptical until see a proof that it, first, works. And, second, that it actually has an advantage over "we'll just train another base model from scratch, but 10% larger, with those +5% performance architecture tweaks, and a new modality blender, and more of that good highly curated data in the dataset, and fresher data overall, and it'll be glorious".
They haven't though. On two fronts: 1, the soa models have been pretty constantly priced, and everyone wants the soa models. Likely the only way costs drop is the models get so good that people are like hey, I'm fine with a less useful answer (which is still good enough) and that seems, right now, like a bad bet.
and 2 - we use a lot more tokens now. No more pasting Q&A into a site; now people hammer up chunks of their codebases and would love to push more. More context, more thinking, more everything.
Here's an analogy you may understand:
Inference is getting so much cheaper that cursor and zed have had to raise prices.
Even if we take this as true, the point is that this is different than "the cost of inference isn't going down." It is going down, it's just that people want more performance, and are willing to pay for it. Spend going up is not the same as cost going up.
I don't disagree that there are a wide variety of things to talk about here, but that means it's extra important to get what you're talking about straight.
The cost of inference -- ie $ that go to your llm api provider -- has increased and certainly appears to continue to increase.
see also https://ethanding.substack.com/p/ai-subscriptions-get-short-...
This is the crux of it: when talking about "the cost of inference" for the purposes of the unit economics of the business, what's being discussed is not what they charge you. It's about their COGs.
That's not word games. It's about being clear about what's being talked about.
Talking about increased prices is something that could be talked about! But it's a different thing. For example, what you're talking about here is total spend, not about individual pricing going up or down. That's also a third thing!
You can't come to agreement unless you agree on what's being discussed.
I think if these companies are gambling their future on COGS going down, that’s a gamble they’re going to lose.
we strive to make machines more efficient to reduce energy usage. That makes $ per unit of work lower. But instead of reducing spending, people find more uses for the machines.
It seems to me like models' capability scales logarithmically with size and wattage, making them the rare piece of software that can counteract Moore's Law. That doesn't seem like a way to make a trillion dollars.
Anthropic: "$2.66 billion on compute on an estimated $2.55 billion in revenue"
Cursor: "bills more than doubled from $6.2 million in May 2025 to $12.6 million in June 2025"
Clickthrough if you want the analysis and caveats
ARR could be a useful tool to help predict future revenue, but why not simply report on actual revenue and suggest it might increase in the next year? I have found the most articles to be unclear to the reader about what ARR actually represents.
The point of ARR is to give an up to date measure on a rapidly changing number. If you only report projected calendar year revenue, then on January 1 you switch from reporting 2025 annual revenue to 2026 projected revenue, a huge and confusing jump. Why not just report ARR every month? It's basically just a way of reporting monthly revenue — take the number you get and divide it by 12.
I am really skeptical that people are being bamboozled by this in some significant way. Zitron does far more confusing things with numbers in the name of critique.
Nobody considers a year from June to June because that would be misleading.
How do you know it's recurring? What data do you have (historic) that makes you believe the revenue will happen again?
Is this based on signed contracts etc so you have some guarantees?
I am also willing to bet that the student dropoff is not pronounced. I am more thinking of a business that sells beach umbrellas, they make a lot of sales in the summer months and then next to nothing in the winter months. That would be dishonest.
so he got a leaked copy of their AWS bills?
Well I don't have to scratch my head any longer and wonder why Amazon hasn't jumped on the AI bandwagon with their own Gemini or whatever. They are sitting pretty and selling shovels and pickaxes to the AI fools. Not a bad strategy for them...
It might take out your 401k for a decade.
At a certain point you just expect it.
[1]: https://en.wikipedia.org/wiki/August_2011_stock_markets_fall
Its like fruit fly generations, not 20-30 year human cohort generations.
I would expect to see OpenAI, Anthropic, and a lot of the little tool wrappers to get taken out though, or at least acquired for pennies on the dollar when it bursts.
But like the last one, it's going to be us, the tax payers, that are left holding the bag.
Amazons investment in Anthropic was in the form of convertible notes, which they have converted entirely into equity by march of this year. At that time, Anthropic was valued at 61.5 billion and Amazon (in their filings) said their investment was worth 13.8 billion, so about 22% of the company.
Then, there was another round in September where Anthropic raised 13 billion more at a valuation of 183 billion (so the new investors are buying about a 7% stake in the company). Without more details, that would lower amazons percentage to about 20% (old investors hold 93% of the company, so Amazon's 22% of the remaining 93% comes out to about 20%). There are probably other details that lower that percentage a bit, but i think the 15-19% ownership estimate is pretty accurate.
Everything else is expense.
There’s a lot of that sort of thing going on at the moment in the AI bubble.
when the music stops, suddenly a lot of people won't just sit on the ground but plunge into the depths of hell.
50% of people into coding agents are quite concerned about that last mile in difference with frontier models that they "can't afford to lose" - my experience tells me otherwise, the difference is negligible once you have a good setup going and knows how to tune your model + agent.
The other 50% don't give a damn, they just landed, or got locked, into some deal for a coding agent and are happy with what they got, so why change? These deals arrived from the big model providers and resellers first, so Chinese arrived late and with too little to the party.
Running Chinese models (for coding) requires many things that you need to figure-out yourself. Are you running the model on your hw or through a provider? Are you paying by token or on a plan? Does the model pair well with you agent CLI/IDE of choice? (Zed, Cline, Opencode, etc) Does it even work with your favorite tool? (tool calling is very wobbly) Is it fast (tps)? Is it reliable? How do you do "ultrathink" with a secondary model? How do you do "large context"? Does it include a cache or are you going to eat through the plan in 1hr/day? What context size are you getting? Does it include vision and web search or do you have to get another provider/mcp for that? And, yeah, is it in a territory where you can send your client's code to? A lot to grok.
Cerebras Coder Max is really cool if you want to hack your way through this, but they couldn't care less about your experience: no cache, no tool endpoint fine-tuning, no plans or roadmap on updating models, on increasing context windows, adding vision, or anything really. They just deleted some of the tools they were recommending out of the website (ie Cursor) as they got reports of things that stopped working.
I am not invested in anything except popcorn to watch it burst;)
Yes you can say you're honest and stuff but does it mean anything if you can go broke from honesty.
that's literally the exact opposite of what i'm saying - literally read my responce to comments up chain.
> i'm starting a new trend: ask every person that is so certain about the negative outlook how big their short position is. so how big is your short position? please let us know.
To me people with short positions likely be bending the truth. To you they are honest. Funny
There's a reason why investing must be declared in many situations. and it is because they are a conflict of interest not proof of honesty
> conflict of interest not proof of honesty
What exactly do you think disclosing a conflict of interest is if not honesty...............
That's the reason in many cases disclosure is mandatory and not disclosing it is illegal. To let people know to take it with a grain of salt.
True honesty = no conflict of interest. Judging things for what they are not what bucks they can make you if you spin it right
Lol bro you're making up the definitions the definition just to suit your weird perspective.
> By definition, a "conflict of interest" occurs if, within a particular decision-making context, an individual is subject to two coexisting interests that are in direct conflict with each other ("competing interests"). This is important because under these circumstances, the decision-making process can be disrupted or compromised, affecting the integrity or reliability of the outcomes.
The decision is what to write on HN and integrity is compromised if "writing truth" is in conflict with "get richer" which can happen if you invest in a sus biz.
We can pretend you are the rare exception that speaks the truth even if it means you go broke but even then it just proves the rule
This is more mantra - professional shorts exist and they make money. So either you know something (in which case you'd be mum about it and just placing your bets) or you know nothing.
No amount of hype from your side can make it a non-bubble market when OpenAI and other companies make wild and impossible to fulfill promises for infrastructure buildup that rely on vague promises of circular money lending between AI SaaS and infra companies.
"The bubble is both obvious and also completely unpredictable".
Makes sense.
But if we lay off the bong for a bit and think soberly, then no, knowing that certain economic process is not sustainable and will eventually collapse, but not being able to profit from it is not the same as saying every bubble would pop at the heat death of the universe, billions years for now. It's so not the same that the very comparison should be immediately perceived as absurd.
You can't both talk about "professional shorts" and use it as an argument against somebody who is not a financial professional. This argument is invalid because not every person who is skeptical about current AI companies' perspectives is willing or able to become a professional short just to make a point on it. One can have a perfectly valid argument and still not be a professional short based on it.
—Kant, Critique of Pure Reason
E.g
What is the unit cost of serving a Token? It is the cost of electricity + amortized cost of GPU (GPUs would have been Capex, but because of their fast depreciation rate, you can claim they should be Opex). Given this cost structure, every SOTA labs (Google, Anthropic and OpenAI) are profitable and actually have high-margins 50-60%.
With this margin and growth, the frontier labs can be profitable anytime they want to. But they are sacrificing profitability for growth (as they should be)
Where is Ed's analysis about this? Either he is disingenuous or clueless. Remember people who voluntarily subscribe to Ed, are coming from wanting to hear what they believe.
If he is level-headed, show me an Ed article that is positive about AI
Why should those two things go together?
But I guess Ed Zitron has found his audience
I happen to agree with the overall sentiment (that AI buildout is overextending the tech sector and the financial markets), but he is utterly fixated on the evils of AI and unable to admit either the current usefulness or the future potential of the technology. This does not make him look like an honest broker.
The rambling nature of his posts also makes it harder to properly argue against them as he keeps repeating the same points over and over; some of them are decent but there is certainly a gish gallop feeling to the whole thing.
He definitely changed his mind on AI coding agents based on reader feedback. Ultimately though, you need incredible productivity growth/massive layoffs to make the numbers work for the current spending and RN, I don't see large signs of this.
> I happen to agree with the overall sentiment (that AI buildout is overextending the tech sector and the financial markets), but he is utterly fixated on the evils of AI and unable to admit either the current usefulness or the future potential of the technology. This does not make him look like an honest broker.
I think this is probably because he feels like he's taking crazy pills when he hears what CEOs/leaders are saying about this. It's some kind of mind virus. Like, I was at a meetup a few months back where a senior data/code person was saying that nobody would write code in 5 years, which (if you've used the tools heavily) seems pretty absurd.
Not necessarily. That METR study was interesting in that participants reported that they were more productive, but the hard data disagreed. This is incredibly common when looking at humans, we're generally bad at knowing what hurts or helps us in this sphere.
And personally, I think LLMs are super useful, but I'm pretty sceptical about valuations and returns in this space over the short to medium term.
A billion people drink Coke every day, yes, and billions also have cavities, obesity, and diabetes.
Majorities also once thought the Earth was flat.
FWIW, I personally think they're correct on both bitcoin and Tesla, but apparently people disagree.
Those people arent exactly experts or right most of the time either
"What is the unit cost of serving a Token? It is the cost of electricity + amortized cost of GPU (GPUs would have been Capex, but because of their fast depreciation rate, you can claim they should be Opex). Given this cost structure, every SOTA labs (Google, Anthropic and OpenAI) are profitable and actually have high unit margins of 50-60%."
High Unit Margins and growth means, these labs can be profitable anytime they choose to
All bubbles (dot com, housing, tech, crypto, etc) have a lot of losers and a few big winners.
That is less a reflection on the market of the bubble and more a reflection of the number, skill and risk taking of the prospectors.
Claiming that a single journalist blog has power to stop others from criticiaing ai for different reasons ia kind of absurd.
I mostly assume that most people end reading his articles somewhere in first third and go on reading something easier to read. His articles are not exactly casual read material and they are loooong. You have to have certain kind of personality to get over first few paragraphs.
I have sat with these numbers for a great deal of time, and I can’t find any evidence that Anthropic has any path to profitability outside of aggressively increasing the prices on their customers to the point that its services will become untenable for consumers and enterprise customers alike.
I suppose that's the pessimistic-on-AI side. On the other hand, once you create God little things like money are meaningless.
Firstly, a huge amount of labour that can be accelerated by LLMs fall into the "bullshit jobs" category, where you can make someone faster at writing emails but the emails themselves don't really contribute much value. The majority of LLM use I see falls into this category. Many people can speed up parts of their job, but you can add as much efficiency as you want without actually impacting the bottom line -- and for various reasons that are not tractable right now, including with LLMs, businesses aren't able to get themselves to remove these roles.
Secondly, the median company is incapable of doing the things that aren't driven entirely by hype or political promises made by executives. We still exist in the universe where they prefer to have all their staff attrition out due to not getting raises, then end up paying the same amount for a bunch of folks that have no knowledge of the business when they inevitably have to replace their best talent.
With all that said, I'm sure a few savvier places would happily drop $1000 month per head if the value is there, but I really think in the average case that this would be more about marketing than any logic. People still buy Informatica in 2025 for much more money than they spend on LLMs.
And secondly, maybe not an important point, but the paid version of O365 is something even my team has been forced to buy. The free versions paywall very important features. The free Word can't delete section breaks!
That said, I hope they're using their prime Visa card so they can get some cash back on that spend.
..oh and this article is free
isoprophlex•3mo ago
swyx•3mo ago
"coming soon" is also really over simplistic. you would have missed some of the greatest tech companies in the past 20 years if you evaluated startups based on their early-year revenue vs infra spend
like sure i have a dog in this fight but i actually want the criticism to sharpen my thinking, unfortunately yours does not meet that bar.
isoprophlex•3mo ago
they spend 104% of revenue on ONE cloud provider and costs scale linearly with revenue growth. assume zitron didn't pull these numbers out of his ass.
educate me how this isnt selling $20 bills for $5. you're a smart dude; i myself aint seeing the "sustainable business practices" here
swyx•3mo ago
pull up Uber's financials leading up to IPO. unsustainable and everyone knew it. they work it out after because they burned money and eventually achieved a sustainable moat. this is why venture exists. HN doesnt like venture which is, well, ironic given the domain we're on.
a better negative argument i'd rather see looks like this - "ive run these aws numbers against the typical spend path of pre IPO startups who then later improved their cost baseline and margin profile and even after accounting for all that, Anthropic ngmi". thats the kind of minimum sophistication you need here to play armchair financial analyst. ed zitron, and everyone involved in this entire thread, incl myself, have not done that because we are lazy and ignorant and dont actually care enough about seeking the truth here. we are as unprepared to analyze this AWS spend as we are to understand their 1b -> 10b revenue ramp in 2025. you havent done the work and yet you sit here and judge it unsustainable based off some shitty "leaks". dont pretend that ed's analysis is at all meaningful particularly because he conveniently stops where it supports his known negative bias.
watwut•3mo ago
jsnell•3mo ago
But Zitron's analysis is as incorrect or irrelevant as ever. The problem is just that as usual there is so much tripe that it's impossible to rebut it all.
The way I'd think about this leaked data is that it's giving us a reasonable upper bound on the fully loaded R&D compute cost for Claude Sonnet 4 - a product that will be making $7 billion in revenue with very good unit economics. Other than that, this just tells us that Anthropic, just like the other frontier AI lab, will be spending as much on training as they can.