The true cost of AI won't be revealed until after a large portion of the customer base has become "hooked" on it.
1. The GitHub Copilot pricing change that started a week ago already change buying decisions.
2. Small name open weight providers selling at what I assume, and hear through grape vine, is a profitable place.
Claude is over priced for what you get, and if headline is true, expensive to run. I do wonder if their API pricing is profitable. That's the word on the street about Big AI, they are making money on the PayGo
I’d say we are at the peak of inflated expectations, exactly where they want things to be for the IPOs as they unload their bags on the public. There will be an absolutely massive crash that will destroy the stock market, you’ll finally be able to buy ram and a hard drive again, and then AI will really take your job in the plateau of productivity that comes much later.
GPUs that can run everything from Crysis to CUDA are a harder engineering problem to solve than creating a chip that's optimized for inference. Not to mention that inference is an excellent first step towards a full, competitive GPU as well.
And it seems all of these advanced chips rely on the most advanced lithography which is tightly guarded and supply locked by a few companies.
My own impression based on inference prices for deepseek or other "open" models in the 1T range (including providers like DeepInfra with no obvious reason to subsidize their API costs) is that Anthropic is offering subscriptions at cost (on average, power users are a bit more expensive, casual users more profitable) and making good profit on API pricing. Profit that then is spent on model training, marketing and development, for an overall negative bottom line
Edit: in case it gets changed: current headine is "Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them"
> Methodology & assumptions: No caching
This is absolutely absurd. I'm looking at the stats for a long session using the API where these states are available and the cache write to cache read ratio is about 300.
I just hope local llms keep getting better and ways to make them run faster on consumer devices improves
There are lots of knobs to dial for your costs.
Our code base is not small, millions of lines of code. It does not take $65 in tokens to solve an issue. I'm running 3-4 claude code terminals at the same time and i'm still pretty close to what it would cost per a token for usage. I don't know what we are doing right with our code or claude.md to make this happen and I don't want to change it to break it.
1. That the API pricing is required to make a profit, rather than being effective market segmentation to make a larger profit.
2. That if subscriptions are loss making, it is not worth having loss leaders.
Meeting next week at the White House by coincidence just before the SpaceX IPO. Message to investors will be dont worry the US has your back...
At which point the corruption is sooo big, that an Empire crumbles under its own stench?
"Trump to meet AI leaders to discuss US investment in their companies" - https://www.bbc.com/news/articles/c98r8r7dz5no
"Trump Officials Held Millions of Dollars of SpaceX Ahead of IPO" - https://finance.yahoo.com/markets/stocks/articles/trump-offi...
Companies may state cash flow positive, operating profit, EBITDA positive, but this is not a true profit in aggregate, just when stripping out many other expenses.
If anyone has evidence to the contrary, please share. Once they go public it will all be free to review, at least.
Input: $0.257
Output: $1.286
Cache read: $0.0257
Cache write: $0.322
Someone or something is having an hallucination that would make an AI jealous
I find it disingenuous when people narrow in and focus on the cost of tokens as if thats the only way the companies make money.
They are doing a massive data grab and stealing and thieving your IP and data, non-training data sharing cannot be opted out of.
That combined with the relatively easy switching costs. It doesn't bode well for AI companies seeking to create a walled garden.
* Hardware improvements will reduce costs
* Model training improvements (read: more efficient model training) will reduce costs
* Better models will reduce costs (more inference for less hardware time while keeping quality constant)
* Tooling and platform will stabilize—less need to dump money into applications and backend systems because they will become mature—also improvements in AI efficiency and quality will lower the cost of maintenance and future feature development
* Energy buildout will stabilize (we will eventually have enough energy supply to meet AI demand)
* Chips market will stabilize (chip supply will catch up to AI demand, lowering the hardware costs)
I’m happily paying $200/mo for Claude code. The tokens I use would be >$10k at API rates. I’m building the best products of my life, and multiple of them in parallel. I’m doing better creative work, finally realizing a game idea I’ve had no time for, etc.
If this level of usage goes to $500/mo… I’ll be out. It’s worth what I’m paying, but hey I went a decade without writing that song or building that game. It’s not freaking heroin, it’s a tool that offers a good value for what I pay.
Could be wrong though.
NotGMan•1h ago
Downside: you need to buy a new one for each model.
Upside: insanely fast inference and zero subscription cost, only one time purchase cost.
Once a certain open source model gets good enough this might become viable.
Right now the landscape is still shifting too fast.
State of the art models might remain on subscription, expensive and might be used by large companies only.
State of the art companies might also create their own hardware with hard-baked weights on chip that they don't release to the public, as it might just make more financial sense long term once they "stabilize" on a certain model.
amelius•47m ago
Retr0id•40m ago
brookst•34m ago
Everything we’re using now is the equivalent of building a GPU on an FPGA: the hardware is general purpose at one abstraction level, and that comes with inefficiency at the next layer up. Collapse the levels, gain efficiency at the cost of generality.
Retr0id•33m ago
To answer my own question, I bet they could figure out a way to still bill you per-token, if they wanted to.
brookst•27m ago
And of course they could bill per-token, same way cable PPV worked (the bits were already in your house). But the cost structure of weights in silicon means that competitors would be encouraged to compete on this per-token cost, as their marginal cost would be zero.
I don’t see that being a durable business model, but I guess the counter argument is it’s also similar to game consoles, where initial hardware is subsidized and the business model assumes ongoing payment for bits.
amelius•18m ago
vitalyan1234•43m ago
the world has long since moved on from that business model, unfortunately.
jodacola•38m ago
Having lightning-speed, local inference of a super high-quality model would be incredible. If you haven't played with it, check out Taalas's demo [1].
Honestly, though - I have my doubts. Recurring revenue is just too nice to pass up; I'm sure AI companies wouldn't want me buying a dedicated Opus card and not giving them money for several years until there's something worth upgrading to.
[0] https://taalas.com/
[1] https://chatjimmy.ai/
jerf•29m ago
Of course expecting the metaphorical Harvard Business School analysts to realize that is asking a lot. Subscriptions are Good and Goodness is Subscriptions, and like any other mass of people following trends the preconditions on when subscriptions are good for a business tend to get lost in the frenzy.
RetroTechie•16m ago
That would fuse 3D graphics and AI accelerators into 1 and the same unit, as far as consumer hardware is concerned.
no-name-here•7m ago