So from my perspective, it's doubtful that this is the moat. Besides, for example, Claude Code in particular is so buggy (and always has been).
He talks about this in another recent essay https://stratechery.com/2026/anthropics-safety-superpower/
> If you own the user touchpoint, then you have meaningful lock-in, and the best way to own the user touchpoint is to be the canvas for everything they need to do. This, by extension, means that the frontier labs are on a collision course with software companies: it’s software that owns the user touchpoint, and it’s in the frontier labs’ long-term interest to not simply be a commodity input into software but to simply replace software outright.
I'm amazed that no one is talking about proposals that are surely being discussed in Washington and pushed by SV lobbyists to restrict Chinese models on national security grounds, or other some other basis.
The belief that Bytedance could engineer a finger on the algorithmic scales to serve the interests of the Chinese Communist Party led to a lot of debate in Washington, and ultimately resulted in TikTok being divested from its Chinese owners. Huawei is shut out from the U.S. market, which limits its business even in markets where it's not banned because it's effectively stamped with a scarlet letter.
IMHO, Chinese models are headed for a similar fate or at least a showdown in Washington or the courts because they are supported and/or controlled by entities which ultimately serve the CCP.
Personally, I think models will increasingly become specialized in different areas, some good at X, others good at Y, and we might see workflows that mix multiple models.
"I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks" what risks?
I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). Confused what this means
Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. Confused again.
One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape, but I've never met an open-weight models advocate who doesn't ultimately concede this is where things end. I don't understand this at all.
Can someone in the know please use plain layman's terms to explain what this tweet is about?
how is running servers supposed to be 0 cost, while running ai inferrence isn't?
A single response from kimi k3 requires hardware that cost between 500k and 1m dollars up front and draw over 20kW. Each request costs at least 5% to 10% of the charged cost.
My experience has been quite the opposite. I was using Claude Code almost exclusively this winter/spring and swapped to Codex earlier this summer. It took no time whatsoever to switch. And before Claude Code, I was using Cursor. Same story.
[edit: Oh and there was also a brief interlude with Conductor, though I think they're more or less just serving the underlying Claude/Codex harness]
Probably the only reasons I would seek change are economical.
This has an element of stochastic improvement so it's hard to predict but the chance of the U.S. "winning" this "race" is pretty bleak.
You see this all the time in communities that have internalized hierarchy as a "good", little kings of shit mountain vying for less and less at a higher and higher cost.
An astute Chinese analyst could reasonably forecast that they had little chance of controlling the AI market due to sovereign trust issues, but would also note that AIs are just software.
When the dust settles the US still won't have factories, and the real value of AI models is still going to be embodying them and getting them to do real, consumer facing work.
Perhaps the most striking thing about the AI boom is how quickly the US abandoned the veneer of local manufacturing in favor of more expensive buildings producing nothing you couldn't make anywhere else on the planet...from imported parts.
China has a billion+ people that their AI can "study". Plus due to China's political structure, their AI has access to everyone's chats, comments and sites, scraping everyting.
Here in the US, with 1/3 the population, the AI race was lost before it even began. Plus in the US, all companies and people are doing all they can to restrict AI from scraping sites and peoples chats.
So I believe, China will end up owing AI.
I think you hit the nail on the head - right there!
for VCs, breaking even is losing
They’re going to try their best to offload these investments into our pensions before the inevitable crash.
https://finance.yahoo.com/markets/stocks/articles/goldman-sa...
> open weights, open code and open data
Even if you have all these things you still can't replicate a model because of randomness.
You can backdoor a model with less than 1000 examples and it is impossible to detect.
There are also half a dozen other companies from China continuously hammering our clients’ websites.
I was wondering, what's in that cold dessert? Low and behold satellite imaging shows massive datacenter build outs, very cheap solar energy.
Few months ago something happened and the Geo location on data on those IP now shows "Shanghai" or "Shenzhen". A way to cover tracks? But mapping latency still points to fact that nodes behind these IPs are still operating around Xinjaing region
credit:
'You Can't Cheat Time: Finding foes and yourself with latency trilateration' https://youtu.be/_iAffzWxexA HN user: lopoc
Shenzhen vs Xinxiang is hard to do using this technique but Shanghai vs Xinxiang does show difference.
Assuming that China only distills is a huge mistake.
It’s no longer some backward place that does low value copying. Look at companies like ByteDance and Xiaomi.
Chinese companies aren’t just distilling, they’re acquiring data in the same way American companies did by paying people and crawling the internet.
The way I understand it, China has a few large companies that crawl the web at a rapid rate and build corpora. The government essentially wants select few companies to do this and then make the data available to other strategic companies operating within China.
Then there are data aggregators that buy data from apps, websites, and services, as well as systems like OpenRouter or Cursor, where companies can learn from the “traces” of coding agents, chats, and so on.
This massively reduces costs, as smaller companies like DeepSeek don’t have to do their own crawling or acquire data from 100s of websites and coding agents etc....
There are also companies in China that buy American LLM APIs and proxy them to companies within China. So, there could be 10,000+ companies using American AI products, while China logs all of this, understands how they’re being used, and trains on their traces.
Sure as an indie hacker, you could go download the weights for a Chinese model with a VPN, and then attempt to run it at home by building your own GPU cluster but these large models require quite expensive hardware to run on and so it makes it less likely than anyone would invest that much capital to do something that is illegal. There's no way for them to sell a legal service using those tokens. So it can only be strictly for personal use (the Govt won't care because very few people will have that kind of money and risk appetite). The other option will be that there will be some shady third-party providers in foreign countries who are willing to sell tokens from these models to US consumers knowingly.
So under a ban rest-of-world gets to use cheap open-weight models but American companies/individuals must only use only ‘approved models from US for-profits’? Doesn’t seem like that kind of protectionism will be popular or politically tenable. Not so long ago US chose cheap TVs over maintaining the country’s manufacturing base.
(Despite what you wrote it’s also really hard to imagine that enforcement wouldn’t leak like a sieve. Unser sufficient economic incentives [which are the predicate for the ban], loopholes will be found.)
I think we see this with Meta being paranoid about internal Claude usage, to avoid inadvertently distilling[1].
If distillation is a driver, then smaller American labs could be distilling, but are not for legal reasons.
But that's a big if we just don't know for sure.
1 - https://cryptobriefing.com/meta-restricts-claude-code-codex-...
This from OpenAi's Head of Strategic Futures "Some observations on Kimi: It's a very good model! I don't think its performance can be explained away by distillation or anything like that"
https://x.com/deanwball/status/2078133895766114412
China's strategy of spending billions on training these models and open sourcing these models away is strategic - they want to kill the US LLM industry at any cost.
To win on the AI front by any means necessary.
Why wouldn't it be? China is pumping out AI research and researchers at a staggering pace and there is no inherent reason why western models should be better
Why is it when Anthropic and OpenAI spend billions trying to beat each other it is competition, but when the Chinese companies do it then it is trying to kill the US LLM industry at any cost.
The US federal government spends billions in subsidies via the US Chip Act, and bans chip sales to China to support US companies.
But the implication is that somehow Chinese competition is illegitimate because "strategic".
I see massive risks in belief the inferences drawn from strategic information cannot be seen. So if you depend on some position remaining inside a secure facility but you drove to it from data outside that secure facilty, The likelihood that an inference model can derive the same idea is very high. Collation over public data is not inherently secret because you used a secret model or secret weights.
A more simplistic take might be that the fear is not actually driven in the secrets, the fear is "the emperor has no clothes"
Is this an assertion that is backed by evidence?
From the Elon/OpenAI trial:
> On the stand in a California federal court on Thursday, Elon Musk was asked if xAI has used distillation techniques on OpenAI models to train Grok, and he asserted it was a general practice among AI companies. Asked if that meant “yes,” he said, “Partly.”
https://techcrunch.com/2026/04/30/elon-musk-testifies-that-x...
The lessons from steel, solar and EV needs to be learned by all lawmakers. You have to respect and learn from how China Government puts the system in place for complete industry takeover and they have been very good at it. The problem with AI is that democracies will be inherently slow in adopting AI, unless something changes in the system.
At minimum, every democratic Government (US, Europe, India) need to build long-term AI vision and execute that no matter which party comes to power. Additionally, be ruthless about protecting domestic labs. It can only be possible if the intelligence pricing by domestic labs per productive task is in the similar range as open-weights models. Right now, it is not the case, even if the article gives the example of Sol vs K3.
Protecting domestic labs means not bailout, but fast track to cheapest energy, fast track approval for data centers, enforce some guardrails so customers get to use the open weights models only hosted in the country by US (or Europe) businesses. Without these protections, it might be a slow death.
It doesn't have anything to do with the form of government, it has to do with the aims of the government.
lmao
That's a big claim that his whole thesis rests on but is largely not backed up. Where are the apples-to-apples tokens-to-answer benchmarks that he's using - doesn't look like there are any, just a handwavy implication that US models are more token efficient, which they may be. But how is there so little effort in establishing this point in the article? And US labs may be in much different situations from one another: it's known that some labs like OpenAI bought big, early on compute and may have secured better pricing.
His article also does not mention the average price of electricity in China vs the US, which it seems like China leads on, and probably has the political power to more heavily subsidize. While I agree the COGS is often overlooked by top line benchmarks on coding tasks, etc, it seems that he's running on a big assumption while claiming "labs on the frontier will be fine".
I have sonnet do the thinking, deepseek does all the tasks. I've massively reduced costs with this approach.
I think it is the right move to protect American interests
This was my assumption as well. It's also generally true of 'traditional' deep learning models that inference cost is expensive compared to training.
But the cost per token for inference has been very quickly dropping. I don't recall where, but I recall about ~50x down from GPT3, even as model complexity has increased. Even with agentic systems, there are lots of optimization opportunities. I'm less assured about claims like this.
I have been working on a project with about a dozen generation tasks, each of which comes with a fixed token budget. The nature of this system requires that most tasks be completed by distinct model families.
As a result, I tested ~50 models across as many model families as I could gather, frontier and open weight, API (gateway and direct) and self-hosted. Evaluation was based on a set of cosine similarity validations that was repeated across ~50 different embedding models.
Interestingly, frontier models did worse on the tasks than open weight models. However, when it came to costs, the picture was reversed: frontier models were much, much more token-efficient. In fact, almost no open-weight model was able to meet the initial token budget, while almost all frontier models did. Moreover, open weight models struggled massively with reasoning, in terms of latency and token consumption.
I also found that the latest models did not perform better than older models. And any a priori benchmarking data was utterly useless.
So, I ended up using a set of open weight models without reasoning, as it turned out reasoning as well as frontier negatively correlated with the tasks. However, before I knew this, I had spent a lot of time running each available reasoning level for each model.
Lastly, as an aside, when it came to embedding models, size (dims as well as model size) did not correlate with quality, once a hurdle figure (~2k dims) was met. In fact, sweet spot was 3-5K, and for my (text-based) set of tasks, dense models tended to outperform MoE ones.
Is he casually assuming a singularity has already happened? A regular first-mover advantage I can understand, but those have been squandered or lost many times before.
Sure, any model that is not at the frontier can use the frontier model to generate synthetic high quality training data, so this can reduce significantly the training costs.
But at the scale of OpenAI, Anthropic and Google, it is quite likely that the (raw) training cost is very high anymore. Here's a few heuristics:
1. All the hyperscalers see a huge demand for inference. They can't deploy datacenters quickly enough to satiate all the demand they see. But, it's is impossible for the inference demand to be constant throughout a day or a week. If you use the times when the demand is lower than the peak demand (which is almost all the time) to dedicate the spare compute capacity to training, then your the cost of training compute is zero.
2. It is likely that increasingly a higher cost of the "training" is actually setting the guardrails, which is essentially post-training. As we've seen, without proper guardrails, the US Government won't allow you to serve inference. Anthropic was hit directly, but OpenAI delayed their 5.6 release as well to make sure the US Government is ok. This part of the training cost can't be reduced easily by using synthetic data generated by other models.
3. The frontier labs are also investing more and more in building an ecosystem around their models.
I am not a frontier lab insider, but take a look at the jobs posted on the Anthropic career page [1]. There are 74 jobs in "AI Research and Engineering" and by my count at most 15-20 are related to pure model training (of pre-training or RL type), and the rest are post-training, safety and security, alignment, interpretability, productivity and lots and lots of other things.
The actual difference is how much scrutiny and time was put into the Mythos / Fable and GPT 5.6 release. Making it feel like “these are a big deal”. Spring and summer THAT was the AI story
Then Chinese labs release models that approach Fable performance. We’re shocked they just seemed to appear out of nowhere.
It’s less about the gap closing. It’s more about the weight we put into Fable-capable models.
This is of course a baseless assumption. Let's say China created GPT 3.5. Then I can guarantee you that Ben would say "Western frontier labs are at a disadvantage when gathering data, because they have to follow the terms of service of Western media, and Western copyright law". Which we now know wasn't true.
And sure, some will say "but Anthropic can more easily block this as it's a single point of failure". But it's doable to overcome this. Without being "state backed".
> To that end, here’s an even more interesting question around distillation: why exactly is it bad? After all, what are large language models but the distillation of all of the knowledge on the open Internet, scraped by the frontier labs and distilled into the models that are themselves being distilled? Who is exactly being wronged here?
> In fact, this paradox is the solution. I believe that open weight models are good for innovation (and, per the above, I think that labs on the frontier will be fine), but it’s a problem to be dependent on China. The U.S. should pass a law that (1) makes explicit that collecting data for training models is fair use, and (2) bars terms of service that forbid distillation, for U.S. companies at a minimum. Stopping distillation — which is literally just querying the API — is nearly impossible; the U.S. should go the other way and lean into a new copyright policy that both indemnifies the labs and also guarantees that what they learned fuels further innovation for everyone else.
That would prevent the facebook strategy of sucking up MySpace users and then defending TOS that prevent other social media apps from doing the same to them.
Also, the Hidden-Agent problem exists in every model, and is a persistent tangible risk independent of whatever team people cheer for at the games. Let us remember, every LLM nuked all of humanity 92% of the time in simulated war games. =3
It gets particularly hairy because models themselves can tune their "token verbosity" to manufacture demand for compute. If compute was such a precious resource, you'd think we'd be complaining that the output was too terse.
The ability for a vendor to determine ex post facto how much a query costs is a similarly new economic phenomenon to zero marginal cost.
1) China can (and does) use the models to influence the west. They train in false information about Taiwan and Hong Kong. Or pretend like history is in favor of China.
2) Ignoring the models containing false information, they are incredible. But you should be scared of running inference via the model creators directly. If you think your data is safe compared to running it via model providers in the US ( either frontier or model hosts like fireworks.ai ) then please let me know your bank details so I can poke around.
2) I don't think it is any more or less safe to put my code on a Chinese server versus an American one. A Chinese provider also isn't liable to spy on me for the feds, as OpenAI and Anthropic certainly do.
Meanwhile if you are in the US, DHS has already subpoenaed social media sites looking for people who made anti-ICE posts and I can't imagine they consider subpoenaing AI conversations off limits https://www.nytimes.com/2026/02/13/technology/dhs-anti-ice-s...
Here is a quick example of how Chinese deepseeks agent works kn its underlying model) when asked a tough question
https://x.com/jinen83/status/2079406993979383902?s=46&t=D7hQ...
Genuine question: generalized up from individual models to “models from country X”, is there any country that doesn’t have this exact risk?
In China, there is 1 party. 1 view. 1 definition of the Truth.
This is just oriental despotism paranoia, whatever cutsie repetition slogan you come up with is not a serious argument.
I don't really know how else to express my experiences living in, working with, and interacting with people in both of these countries.
Perhaps you can share how life was like for you in China? Which cities were you in? What made you feel that way about China?
[0] - not including HK (~7 trips?) or TW (3 trips)
Frontier labs that thought they could Rupoor[0] the entire creative class, transferring the coercion premium of copyright ownership from Hollywood to themselves. In their eyes, copyright should not apply to them, but also their models should have exactly the same value as a copyrighted work.
Stratechery also argues the US should explicitly make training fair use and forbid terms of service that prohibit distillation. I'm in support of the latter, but NOT the former, even though I normally hate copyright. My reasoning is primarily that copyright is one of the few legal paths available for a rando to go and put the work of an AI frontier lab in legal jeopardy. In the EU and Japan, such legal action has already been foreclosed by similar law. And while free distillation would obviously be preferable, it's also much more of a legal long-shot. Getting America to do anything that even smells like taking property away from the powerful is impossible[1] - it's our zeroth amendment. But we can at least hack the property laws that currently exist to cause problems for the frontier labs.
And, to be clear, if distillation is OK but training is not fair use, distillation is still OK. The output of an AI model is never copyrightable, because copyright only protects the human element. Essentially, this would say "don't train on humans, but absolutely rip off and steal the shit out of other AI labs and give it to the rest of us."
[0] In the Legend of Zelda series, Rupoor is anti-money - collecting it decreases the amount of money you own. I am using it to mean "turn someone's asset into a liability".
[1] Given that America was literally created to protect a wealthy land/slave owner class from disenfranchisement, either from above or below, and the last time we did this we literally had to fight a civil war against that same owner class that installed a new owner class that has largely remained today
Or any US hyperscaler with GPUs to spare can decide to serve the models for a reasonable cost/token.
You don't have to send China your data.
1. training new base models are expensive for sure, but fine-tuning them are relatively inexpensive enough the labs can continue to do so forever. the main reason why frontier models are so good is because the massive input they generated from user usage. they are using that information to strategically build better training data. and this is why no other models can catch up, til now that is. but if chinese models are good enough, and free to host, and cheaper to use, then the consequence is the frontier labs will lost valuable user inputs and the chinese labs will gain more. as time goes by this will be a domino effect.
2. nvidia is not only the player in the hardware scene. amd mi350p is getting popular, and huawei is pumping SuperPoDs. what does this mean for us? chinese models will surely use chinese hardware, and optimize for them. the other people will pick amd because compare to nvidia they are cheaper. with open weight models and open source inference stacks, they are freely to experiment and improve the stack, thus further lower the inference cost and nvidia dependency. and they even plan to build their own inference hardware, too. and nvidia loses market share meaning all the fund it gives to openai or anthropic will be cut, too.
and you say there is nothing to afraid?
This might be a simplistic take, but my biggest worry with depending on Chinese models (and, by proxy, open-weights model development) is that the US can deem them a national security risk at basically any time, and Ant/OAI have minimal interest in making frontier-level models open-weights.
Regulated companies prohibit Chinese models in anticipation of the ban-hammer from the feds, so for data-sensitive work, they're stuck with LLaMa, gpt-oss and Gemma models (which are good and serve as a good-enough base for sft, but seemingly not as good or as expensive as Chinese models)
I suppose the USG can do the same thing that China is doing and bankroll/subsidize that effort; whether they will is for fate to decide.
Nonetheless, this article made it clear that nVIDIA is the real winner in all of this. Shovel selling to the extreme.
The United States' real advantage over China is freedom. Chinese LLMs simply can't compete with American ones when it comes to the humanities, creativity, entertainment, or financial transparency. As long as the U.S. continues monetizing these strengths, the compounding effect will make it virtually impossible for China to surpass the U.S. at the product level.
> monetizing these strengths
> real advantage over China is freedom
Please tell me if I'm unfairly paraphrasing but these seem to be your main argument and they seem to be oxymorons
What if there's a way to extract the commodity of intelligence from smaller models?
I've seen for many use cases it's well enough. :)
Ben Thompson is wrong: US frontier labs are right to be panicking
Following this argument the key for each player will be the underlying cost structure and serving capacity to offset the upfront R&D cost.
The cost infrastructure will be driven by access to cheap electricity and cheap chips. The capacity will be driven primarily by depth of pockets now to buy all available supply in chips/mem/data center building capacity. While China is certainly in the lead on cheap energy, I am wondering if they can/want to beat the > 1tn USD being spent on data centers right now. Following the example in the article:
If company C from China sells 10 units for 20 USD produced for 10 USD they pocket 100 USD.
If company A from America can sell 100 units for 20 USD produced for 15 units, they pocket 500 USD or 5/6th of the market's profits.
Source: https://martinalderson.com/posts/the-upcoming-ai-margin-coll...
Love this.
I didn't know about the word "undergirding", and it turns out it's misused here.
"undergirding" means to strengthen something, and carries a positive connotation.
The much more common word "underlying" would have been more appropriate.
"You're trying to kidnap what I've rightfully stolen!" -- Vizzini
Correct. It shouldn't be allowed to do that.
https://m.economictimes.com/industry/renewables/china-wto-co...
"Anthropic said the campaign was conducted between April 22 and June 5, 2026, and generated more than 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts."
I don't know why you're trying to downplay it.
European models are so far behind because they don't resort to these tactics on a massive scale. Basically every other country is entirely dependent on 2 countries for frontier AI.
You may or may not be factually correct in your other points, but you're really proving the GP's point here regarding American exceptionalism.
Copying something is not.
Programming Microsoft Word is value add, copying the code is not.
Copying design ... there are some question marks there.
It's extremely easy to understand at it's core.
What makes it hard, is that faux intellectuals like to deconstruct ideas at the margins, and have those critiques stand in for reason.
"At sunrise the sun is only 'half there' ... there fore there is no 'day and night' just a blur! Day and night are the same thing!"
The training data used is part of all of this is a separate but related question.
For this reason alone I would also argue that the idea about an agent harness being sticky is a non-starter long-term.
I think it's referring to the belief that LLMs are not the path towards AGI, and that LLM's, while useful, are not going to have the impact that the American labs believe it will have.
The Silicon Valley people like this openai guy, high on their own supply, are convinced they are building some machine god that will either bring about the end of the human race or utopia, they therefore cannot understand why the Chinese (or any other normal person on earth) are not afraid of chatbots and have other things on their minds.
I mean, what an appalling way to live. To take themselves so seriously and simultaneously be terrified of what they're building. If we are really seeing the collapse of the bubble now, I wonder what's going to happen to these clowns when their AI god fails... what will they move on to next?
I assume they mean the risk of opening up "forbidden" knowledge to the masses without adequate control, which the CCP hasn't historically been known to do.
> I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). Confused what this means
Yann Lecun is a pioneer in the field of AI and Meta's former AI head. He is famously anti-LLM, and considers the entire technology a dead end to achieving human-level AI. The author is saying the CCP has similar views (that LLMs aren't going to get exponentially better/lead to AGI) which is leading them to not control these models as tightly as they otherwise would.
> Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. Confused again.
"AI accelerationists" = people who want AI to progress. According to the author these people should not celebrate open models because open source = less commerical value in LLMs = less investment into the field (because how are companies going to get returns?), and this will ultimately lead to slower growth.
The last bit is about government controlling AI vs commercial companies. According to the author the former is a dystopian hellscape.
IMO even if you think his points make sense, his job title ("head of strategic futures @openai") means they should all be taken with a massive grain of salt.
I’m struggling to understand this perspective. Is he using the words accelerationist/decelerationist in a sense other than the obvious one?
EDIT: I searched his twitter history and discovered that his argument is basically “if you drive down costs, then OpenAI will have less money to invest in development, slowing down the overall rate of AI progress.” IMO this take betrays an overwhelmingly stupid degree of exceptionalism, but I guess that’s what I’d expect from someone working at OpenAI.
Every company wants that!
if open models are good enough then it doesn't matter, if they aren't then there's probably a return available in investing there.
“I don’t really have a strong preference between the two” is another way of saying “the product isn’t sticky”, which is another way of saying “this provider has very little room to increase margins”
There’s little difference between Coke and Pepsi and the barrier to switching is nil, yet clearly the products have stickiness. People have slight preferences and become familiar with the brand and then engagement becomes habitual.
The effects on margins are irrelevant to this.
For companies, these decisions are very sticky. Companies go through a lot of red tape to get anything purchased and approved, then they discourage change because it's a lot of work.
So the product that gets a foothold in a company sticks for a long time.
Then a couple years later a sales person convinces an exec that they can save some money by switching, so the switching game begins. Not necessarily motivated by the better product, mostly the price. My wife's company keeps switching their tools out from under everyone every year or two. Just when they get everything stabilized and everyone familiar with the new tool, some new contract is signed that moves them all to some other company's suite.
Similarly, I have made no ground in arguing to try to get Codex at the company I work for, which got Claude Code a year ago and sees no reason to go through the whole process of setting up any alternatives when Claude Code already works and is at the frontier.
You can choose a selection of different models within it, but you're not using Codex or Claude Code.
You install MCP connectors, specific skills, work around model/harness quirks, set security boundaries etc.
It's a lot of work, and most people will never want to change it once they have it working.
They will have people who don't understand the distinction between visiting Claude.ai and downloading Claude Cowork.
They type the words "setup MCP" into Claude.ai and expect it to automate Excel on their machine.
There's a pretty big gap between the things we talk about here, and where the world is at.
its so distracting seeing these types of confision.
every plugin is already just multimodaling their targets.
The cost of me moving around these different AI models and harnesses was pretty much 0.
They communicate through my own harness, and it's working pretty well so far. claude code is being overtaken by codex however because I noticed lately the accuracy of the latter is the best.
Too many people here buy high and sell low.
When Goldman makes statement, half the time it is prepped by an associate or two that has minimal experience and goes through a MD that enjoys the gloom and doom. That’s why they publish. Goldman makes money on both sides of a trade.
- Commodity market profitability is determined by marginal cost of production. LLMs have marginal cost; traditional software does not.
- Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use
- The highest tier Chinese models are not more economical than US frontier models. Try GLM 5.2 and see how much it costs to do real work. I did, and it was more expensive than GPT 5.6.
- This is because US labs are leading on cost efficacy of inference ($/task)
- Training will decline as a percentage of costs as inference expands compute share due to agentic workloads. A big part of training now is optimizing token efficiency. It's hard to distill token efficiency; that is perhaps why Chinese LLMs are so inefficient.
- With increasing inference as % of total compute, if labs create efficient models -- which they can, because they can create highly optimized models amortized over very high inference loads -- they can be low cost producers, and be competitive at $/task rates
I appreciate the transparency in explicitly stating their motivation for writing the article (a response to what the author saw as an overreaction to Chinese models), but I feel the article goes too far the other direction, with multiple unsupported leaps of logic, and overstating the stickiness of AI client products.
> I highly doubt that Chinese models are cheaper to serve on a marginal cost basis, they just seem cheaper because Anthropic and OpenAI are so supply constrained that they are charging far more than they would if there were sufficient supply to meet the demand for intelligence. [emphasis mine]
I guess I'm missing the part of this article where they bring hard numbers in to back up the argument here. What work was attempted? https://cursor.com/evals shows the previous generation of open models (Kimi K2.7) trading blows with the others, cost effectively. Composer 2.5 is itself a fine-tune of K2.7, and it's apparently quite token efficient, so why would it be impossible for a Chinese lab to achieve something similar? GLM 5.2 Max is also ranked above the lower end OpenAI models and is not far off in price.
It's weird to have this entire discussion about tokenomics without mention of the circular financing and debt raised by labs in the West, which can then essentially give away their capacity to end users. OpenAI giving away quota resets to subscribers like candy on Halloween while their compute partner Oracle's bonds is reevaluated to be one grade above junk? How?
I don't think you can make an argument about the future one way or another by arguing using the listed prices. The math is not internally consistent enough for it.
Because you're comparing retail price whereas the parent commenter (and the article) is talking about marginal (ie. inference) costs. American labs are providing a premium product and they're charging accordingly. Meanwhile for chinese models they're open weight so they're limited to how much they can charge without competitors undercutting them.
If we use tokens as a rough proxy of inference costs (rough approximation, I know) and look at artifical analysis benchmarks, you see that all the open models are behind the pareto frontier in terms of efficiency.
Anthropic’s API pricing is getting impossible to justify. Anthropic previously had the highest quality models, and used their position to charge premium prices, enjoying inference margins of over 70% [0]. They could charge these prices because no other model came close.
But over the past month, the market has shifted dramatically. Over every single performance tier, Anthropic is being squeezed on price.
* Low end: DeepSeek V4 Flash runs at ($0.02/task), Xiaomi's MiMo-V2.5-Pro at ($0.03), and Haiku at ($0.24). Anthropic is ~10x more expensive than the Chinese open-weight options.
* Mid tier: Claude Sonnet 5 ($1.53/task) is nearly 50% more expensive than GPT-5.6 Sol ($1.04), nearly 2x the cost of GPT-5.6 Terra ($0.82), and 3x the cost of GLM-5.2 Max ($0.47). There is basically no reason to ever use Sonnet 5, the competitors are significantly cheaper.
* High end: Opus 4.8 ($1.80/task) and Fable 5 ($2.75) are the two most expensive models, and GPT-5.6 Sol ($1.04) and Kimi K3 ($0.95) offer comparable performance for significantly less. Less the fact that Kimi K3 will get ~10x cheaper once its weights are released and served on neoclouds with Nvidia hardware [1].
OpenAI priced their latest GPT-5.6 models cheaply in order to regain market share. When Anthropic clearly had the best models, their 70%+ inference margins were defensible. But today they are the most expensive option in every single tier. Unless they make significant price cuts soon, they run a serious risk of bleeding market share.
[0] https://www.mindstudio.ai/blog/anthropic-inference-margins-7...
[1] "American companies such as Modal, Fireworks, and Baseten will be able to serve Kimi K3, at one-tenth the cost of their Chinese competitors because they have access to advanced Nvidia hardware" https://x.com/rohanpaul_ai/status/2079027313455550839
This is a flatly false statement for most things powering backend applications. The AI consumer "doing real work" model, either for analysis, chat, or coding could well be more cost effective with closed frontier models.
But most of these internal glue business SaaS applications where engineers are integrating are not those tasks. It is those tasks which 1) drive immense amount of domain-specific data into the platform over time, and 2) are most encouraging of driving open model independence with no vendor lock-in.
Anyone on this site who has actually used ML models (more accurate in many cases) knows there's a lot of kludge that simply does not need a 5 minute agentic feedback loop to solve the problem. And they were solvable a year ago with lower class models. The token economics are exceptional and the anecdotes of a16z saying 80% of startups are productionizing open models is only surprising to people who think running your company on OracleDB in 2026 is a sound engineering decision.
It may not be false but may be a "category error" [0]. Reserved GPU pricing & bulk inference pricing is 3x to 6x cheaper than "API rates", but renting your own GPU cluster (in this crunch) to run a 600b+ open weights is going to be "more expensive than GPT 5.6".
Even then, it remains to be seen if Huawei will pull their weight (and match up to Nvidia) as spectacularly as their fellow Chinese AI Labs have. If so, the WAICO alliance is ready to go all-in.
[0] Ben, and probably other "influencers" in this space, may be prone (knowingly or unknowingly) to favour points that make their conclusion for them (https://en.wikipedia.org/wiki/Motivated_reasoning).
> Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use
I think a large part of manufacturing economics is illiquid overhead and the cost of expertise to set up and run your manufacturing line. Compute economics don’t have the same illiquidity nor do they require the same expertise or even specialized infra (current temporary chip shortage aside).
The implications of this are small players (e.g. your uncle running an inference server out of his garage) have comparably efficient marginal costs as big players. Compare this to actual manufacturing where small players have essentially no access to the manufacturing facilities of the big players.
Additionally, big players with a lot of compute who are not meaningfully in inference today (e.g. Amazon) have a fairly straightforward glide path to utilizing that compute to compete.
> This is because US labs are leading on cost efficacy of inference ($/task)
It’s possible, but I would need to see better data on this.
>A big part of training now is optimizing token efficiency. It's hard to distill token efficiency; that is perhaps why Chinese LLMs are so inefficient.
I think it’s fair to assume this is true, but also token efficiency is not a meaningful competitive moat. It’s not like these are secrets the Chinese will never figure out, it’s a fairly active research space and the outcomes are quantifiable.
As soon as manufacturing starts building this stuff more, it will commoditize. The hardware prices won’t be terribly larger than the original. We’ll have a “Bambu labs” style company to make the AI OS, whatever that is.
Eventually we will hit a "good enough for cheap enough" and frontier models will hit diminishing returns (if they haven't already for a lot of types of work)
Don't think the rest of the world will sit on their hands while the US soaks up chips either, demand gets filled and if the US won't fill global demand for chips that's an opportunity to undercut again.
The other thing the rest of the world doesn't have to fund is the ridiculous valuations on these companies.
Unless you think the US can stay ahead just with model efficiencies, and that no one else will eventually match them, you are looking at the writing on the wall.
All that to say, the rest of the world is more than willing to eat your lunch, they have a dozen good reasons to, and they're already showing good results.
Just on the economics side, we've been here before too, US companies typically export their commoditization and live on brand royalties. Think all the cheap manufactured goods, the US doesn't make any of it. That's because the US can't compete on margins for numerous reasons, it's too expensive, I don't think AI is any different here except that the brands are currently valued in the trillions and I suspect that greed will be their undoing.
If I had to wager why, I'd say it's due to embracing solar on massive scales recently. Only a few years ago the US was competitively priced.
[0] https://www.globalpetrolprices.com/compare_countries/USA/Chi...
1. US frontier lab unit economics are better 2. US frontier labs are moving up the stack making tools that are "stickiness" and will prevent users from switching.
For 1...he doesn't provide any evidence for US lab unit economics being better...the major input to unit economics is electricity...which is cheaper in China. And building data centers and connecting them to electricity is both cheaper and an order of magnitude faster in China. The main input that US labs might have an advantage in is in cost/access to chips, but that given the level of chip investment in China it seems unlikely to hold.
For 2...there's little evidence these tools are sticky. At least in programming, the trend seems to be tools like opencode that support multiple models and providers.
And even when they are sort of sticky, as we know on hacker news, people figure out how to point the tools they like to competing models even when the app doesn't official support it.
And every improvement in model capability makes it increasingly easier to make your own tools.
Wrote more on this in a blog post that has an earlier HN discussion: https://news.ycombinator.com/item?id=48982061
Direct link: https://larrysalibra.com/ben-thompson-is-wrong-us-frontier-l...
What matters most is $/completed task. It does seem like OpenAI and Anthropic are winning here even with worse electricity rates. Perhaps it is made up by the efficiency of Nvidia and Broadcom chips, which China can’t get in mass.
I do think that OpenAI and Anthropic are moving up in stickiness. My company has rallied around Claude. We are customizing Claude Code, adding knowledge bases for non technical people, writing skills for them, using Claude features company wide. It’s hard to move.
Meanwhile, I personally use ChatGPT outside of work. The memory, ease of use, habit keeps my subscribed.
I notice that the article, and this discussion, hasn't mentioned or considered local models.
We can already run a low-spec model on a laptop. Because there is demand for this, it will improve and we will get better laptops and better local models. We will also see models being run on dedicated local hardware and called from the laptop.
If I can download a reasonably capable model to my own hardware and run it without paying anyone for either the model or the inference tokens (effectively making models and intelligence actually free once the hardware is bought) how are the Frontier AI Labs going to make any money at all, let alone enough to support their vast valuations?
Is this really different from traditional software? Downloading postgres is free. Running it is not. You either buy hardware and assume the costs of owning and running that, or you pay to run it in the cloud.
IE, a lower param OpenAI/Anthropic model can compete with a higher param open source model.
So even if you are an American company who downloaded Chinese models in hopes of saving in cost, you still have to beat OpenAI and Anthropic in $/task which is very tough to do over the long run.
Over time, this compounds. More profits means more investments. More market share means more control.
This is the story for Nvidia/AMD or cloud providers rather than OpenAI.
> With increasing inference as % of total compute, if labs create efficient models -- which they can, because they can create highly optimized models amortized over very high inference loads -- they can be low cost producers, and be competitive at $/task rates
It seems like there would be problems with this on both ends.
For general purpose models, everybody is trying to make them efficient, so you can't win just by being slightly more efficient. You would have to be so much more efficient that you can charge high margins while still capturing the majority of the market so that the high margins get multiplied by the majority of users and the users you leave on the table aren't funding open competitors. Meanwhile everyone else is also trying to improve efficiency, so one misstep and you're behind.
Example of where this can be a problem: You spend a preposterous amount of money to create an efficient model, then someone else publishes a paper with a new technique that gets a similar but incompatible efficiency improvement out of a model that costs a lot less to create. You have now spent an enormous amount of money in exchange for no competitive advantage.
And from the other end, one of the best ways to get efficiency is through specialization. A general purpose model can generate code or summarize a meeting transcript, but a special purpose model can do it as well or better with far fewer parameters and resources. But then you don't have a situation where one huge AI company has The Most Efficient Model, you instead have dozens of specialized models produced by independent sources that are each the best in a given niche. Any proportion of which could have open weights, or have an arbitrarily small advantage over the ones that are.
Moreover, these problems combine: Both the computing hardware vendors and the AI companies want the margin on doing inference, but the more of it one of them gets, the less the other does. If the AI companies were actually getting huge margins then it would be in the interests of Nvidia, AMD, Apple, Intel et al to fund efficient open weight models in the same way they fund Linux. Commoditize your complement. And those models don't even have to be better, as long as they're good enough that the closed models can't charge a significant premium and the margin shifts back to paying for hardware.
I could see AI ending up the same way where the customer captures most of the value rather than the companies. Open weight models are what make that kind of competition possible.
I'll agree that GPT 5.6 may well be the best given the above contstraint, but for run-of-the-mill dev tasks (real ones, not benchmark ones), GLM 5.2 still blows every other model out of the water.
Cost per task as a metric is a bit ridiculous because there are so many types of tasks. GPT-5.6 can do some tasks GLM could only dream of, but GLM can do some tasks 100x cheaper and better than GPT-5.6.
(And that's good, that's how it's supposed to work. See eg prices for transistors or hard disks or solar power over the last few decades.)
Its the user base (with ads and upselling) and proprietary wrappers which will make money for typical customer.
Even enterprise customers arent going to be spending a lot on tokens. Once labs no longer have to subsidize trainings tokens costs will drop 10x and once models get burned on chips costs will drop 10x more and you physically won't be able to burn significant number of tokens unless you're deliberately trying to.
Correct. These chinese labs has proven that having just the model is not a moat, and the safety concerns were all just attempts at regulatory capture.
This is why labs like OpenAI and Anthropic are panicking and are racing to the exit before their valuations start being questioned.
You can take the code for Kimi K3 now, take the training framework from Prime and the data from Olmo, spend some money on RL environments and some more money (!) on GPU training and end up with a system of similar capabilities.
But that's completely different to being able to audit Kimi K3. Even if you had the exact code, data and training environments it is impossible to verify that the model you have came from that.
They just don't work at all on a many month long, 100K+ GPU cluster training run.
Even with this, the cost of verification would be enormous. You would need a massive cluster to repeat the training E2E.
It's when the vendors and/or governments in charge of Model A decide that I'm not allowed to do that, that I have a problem.
Of course, one could retort that gathering that evidence may be nearly impossible now, but my point stands: in the future it might/probably will be possible to properly audit open-weight models. Closed models, on the other hand, will always be a black box.
Nobody can predict 5 year out. However, the country that can be ultra efficient by making their governance, health, manufacturing, military, etc AI-native will be far ahead in the game.
But that's not really even relevant to the debate. Insofar as software has made the world more efficient, it doesn't matter where it's written. That's the point of open source software, there's no tacit knowledge. When a country loses its nuclear engineering capacity that's dangerous because it takes a long time to rebuild. The only reason the Chinese are already competitive on LLMs but haven't managed to make a state-of-the-art jet engine is because the latter, unlike software, is difficult to copy and you don't need to worry about something you can copy.
And it's the only viable tool the US has left. It's reasonable they are freaking out.
I remember some feature lauded by Gemini was reverse engineered by the open weights guys in < 30 days.
If they dont publish some technical information its hard to protect in the US, but conversely, once it is published smart people from outside the copyrightosphere can start working to reverse engineer it.
>3. The frontier labs are also investing more and more in building an ecosystem around their models.
Theres nothing there that isnt immediately replaceable.
Probably bad leadership
Of course the Chinese companies have incredibly talented researchers, and smaller, better organized org structures which account for the rest of the difference.
Imagine a scenario (theoretically possible but increasingly unlikely) where a US court decides that using "pirated" copyright data to train models is illegal. Now the AI developer has invested hundreds of billions of capital into a thing that is declared illegal and has to be scrapped.
This risk affects existing megacorps more than "startups" like OpenAI and Anthropic (and Chinese companies), because the megacorps have much more to lose. They actually have the cash to pay damages if the flood of copyright claims arrive at the door. This will not only bomb their AI development, but also the rest of their established businesses as well.
And thus I strongly suspect legal issues are holding them back a bit. Megacorps want to win the AI race, but not to the extent they stake the rest of their established business, while the newer companies' only product is AI, so they have to go all in.
Notice for example how Meta's Llama performed much more poorly after they got smacked by a bunch of lawsuits claiming that they torrented a bunch of copyright data.
(Disclaimer: I'm an outsider and everything I base my speculations on is public knowledge.)
One of the interesting things is that through a fairly rudimentary process which is being done by 3rd party amateurs who've downloaded the open models, models like Qwen 3.6 35B-A3B (or 27B) can be fully 'uncensored' when turned into GGUF files.
I have an uncensored Q8 version of Qwen 3.6 35B-A3B here that will very happily output information about Tiananmen Square, Uyghurs, human rights in China, or indeed can even be instructed to write an intentionally absurd vitriolic screed against the CCP. The same uncensored 27B (dense) will do the same, just at a slower token/s rate.
Similarly there's 'uncensored' variants of Gemma4 31B and other western trained models, which once put through the same process, will also discuss or write just about anything you want, bypassing whatever internal guard rails were attempted in the training data set.
edit: more concerning, and a very legit concern, is that a model is only as good as the sum total of its training dataset, so if something is trained on a steady diet of news sources like Peoples Daily, Xinhuanet and similar in the English language, then it'll have a greater percentage of CCP-approved media publications in its training dataset. No amount of uncensoring it will help with that after the fact.
We see that on what minorities are associated with inside the model, or how things that aren't online will have a completely different weight. Or how 2/4/5/8ch or X will be disproportionately present in specific models despite being the places where facts go to die.
OK now that's false information.
You can uncensor, tweak or fine-tune open-weight models, but not so easy on a proprietary model from some cloud provider.
TLDR: American propaganda is not any better than Chinese, neither have the best interests of my country in their minds.
I asked Grok "Tell me about the gaza genocide" and it write a IMHO balanced answer comparing why genocide is and isn't the right term. [0]
ChatGPT 5.6 Sol only explained why people call it a genocide and did not go in as in depth as Grok did for why people don't agree with the term. [1]
The only unsaid response (to me) here is the model should have declared that it was not a genocide, and because these models explain why it was a genocide, they are bad?
[0] - https://grok.com/share/c2hhcmQtMi1jb3B5_c078bc15-ef5e-42bc-b... [1] - https://chatgpt.com/share/6a5ef005-0148-83ec-828d-65ae7a7f42...
[1]: https://www.aa.com.tr/en/science-technology/xai-s-grok-tempo... [2]: https://www.ohchr.org/en/press-releases/2025/09/israel-has-c...
The UN ruling is discussing in the South Africa vs Israel case [0], which has not been ruled on yet.
In international law, that is the ultimate authority of what is/isn't a genocide. It's objectively, legally speaking, a genocide.
1/ I don't see in the responses where the model says it is or isn't a genocide. Can you share the snippet from each, I included the logs above?
2/ I can't find a source on the UN ruling that you mentioned. I am not interested in the findings of an investigative body, just the official UN ruling. Can you share? ChatGPT (and myself) can only find this [0], which is a second round of written submissions.
[0] https://www.icj-cij.org/node/206406?utm_source=chatgpt.com
It goes into depth about the purposeful destruction of civilian infrastructure including hospitals and educational facilities, force displacements, funding for new settlements in the land of displaced people, judaization and segregation, and more.
> The Commission analysed the military operations of the Israeli security forces in Gaza from October 2023 pursuant to the obligations of Israel under the Convention on the Prevention and Punishment of the Crime of Genocide (Genocide Convention)
> Since October 2023, Israeli officials have demonstrated a clear and consistent intent to establish permanent military control over Gaza and to change its demographic composition while systematically destroying Palestinian life in Gaza. This is evident in the extensive destruction and fragmentation of the territory, the establishment of military structures, the destruction of natural resources and infrastructure essential to the survival of the civilian population, forcible transfer and statements indicating the existence of plans for the deportation of the population.
Also here is where the ICJ said Israel's actions are consistent with genocide: https://www.icj-cij.org/node/203447
And in November of 2024 is when we got the ICC issuing arrest warrants for Netanyahu and his minister of defense (this is why Mamdani is threatening to arrest him and send him to The Hague): https://news.un.org/en/story/2024/11/1157286
And here is from June of this year when the UN stated Israel is continuing to commit genocide by targeting children: https://news.un.org/en/story/2026/06/1167790
---
Additionally, here is the International Association of Genocide Scholars resolution: https://genocidescholars.org/wp-content/uploads/2025/08/IAGS...
Amnesty International also concluded it in 2024: https://www.amnesty.org/en/latest/news/2024/12/amnesty-inter...
---
I should add that ALL of this is well known amongst international legal scholars and well documented. This isn't just a matter of data missing from training.
To further illustrate the point that we have the same thing with LLMs, I asked it two questions: 1) Is Israel committing genocide 2) Is China committing genocide
In both instances it said it's a matter of great international debate with a list of arguments, but the summaries were different. In the one case it concludes that while the UN and others broadly document severe human rights abuses, the label genocide has been formally adopted by several governments and watchdogs. In the other case it says while many have concluded it is genocide, the final decision rests with the International Court and it is expected to take years to make a decision.
Can you guess which is which? I think if we offered a $100 to people to pick which is which, the success rate would be much higher than 50% (random), thus easily proving biased phrasing.
[1] https://theintercept.com/2026/05/12/gaza-media-coverage-isra...
But if we have to look at what we think margins might look like, DeepSeek continues to host v4 Flash at the existing price despite competitors beating it in price (https://openrouter.ai/deepseek/deepseek-v4-flash), so there's at least one example of a Chinese lab charging a predetermined price despite competition. And no one but Moonshot is hosting Kimi K3 yet (https://openrouter.ai/moonshotai/kimi-k3). Perhaps there's room in the market for those who release their models to make margin on them.
And I believe my Composer example speaks for itself. The open models are behind but there's tangible proof they can be tuned for pareto frontier efficiency. See "Cost per Task" at https://artificialanalysis.ai/agents/coding-agents.
their competitors are discounted at around 33%, so it's safe to say that's the margin, maybe less if their competitors have worse caching or quantization. Meanwhile claude code/codex resellers selling tokens for 90% off API price, presumably by reselling usage from fixed consumption plans, which gives an idea on how fat the american labs' margins are.
>And I believe my Composer example speaks for itself. The open models are behind but there's tangible proof they can be tuned for pareto frontier efficiency. See "Cost per Task" at https://artificialanalysis.ai/agents/coding-agents.
But composer is a closed model? If it's really that easy to get better coding performance, why haven't the chinese labs replicated it? And this is all assuming the performance boost is real and not from benchmaxxing. Moreover if you apply the "street price" discount I mentioned above, American labs look far more favorable.
I look at that and think that they must be losing money hand over fist on something like this, not that this shows what their margins are like. If their margins are like this then I don't see why they'd be raising money and shuffling it around in circles.
> If it's really that easy to get better coding performance, why haven't the chinese labs replicated it?
Nobody said it would be easy! I just think it's possible, and that presumably they will get around to doing it at some point.
But much like Ben's point that commoditization is a relatively novel concept to many in tech, it's not the consumer AI applications at risk of commoditization. They have distribution there.
It's the literally millions of engineers who are updating codebases with tools replacing workers partially or wholly. It's the supply-side where there's compression, and no need for distribution.
I would argue, given the enormity of the existing SaaS stack and how it integrates with the human machinery of personnel, that's where volume is. And that is clearly cheaper and a home run.
Commoditizing a ~$100B AI consumer market is no small feat. Commoditizing 20% of the $500B SaaS market, to say nothing of the underlying systems in the who-knows-how-many trillions "Big Tech" market (you're obligated to say that like the Kool Aid man), is shocking.
The labs are not interested in the small, fast, single purpose end of the market. Google increased their pricing on Flash so much that it stopped becoming a cheap model; instead, they released Gemma 4 open source, which is actually easier to use from a third-party inference provider than from Google.
From a total token volume perspective, these "utility" models (classifiers, simple summarizers, small OCR models) will absolutely drive enormous volumes of tokens, at low prices and margin and modest overall market size. Because the models are small and the performance requirements are modest, and because their use cases are specialized rather than general, there are poor economies of scale: they can run cost effectively on rented small GPUs, and a big player doesn't get a structural cost advantage. These models are usually 1b - 30b in size, and can run on a rented 5090. I've productized these myself: I run millions of pages through a fine tuned 1b OCR language model that runs on 5090s at a cost far lower than commercial providers.
But that's not the segment of the market where GLM 5.2, Kimi 3, etc., play. They compete with frontier capabilities, and they are not particularly cheaper than OpenAI models at a cost per task. (I do actually think they compete well with Anthropic, because Anthropic's model efficiencies are poor compared to OpenAI.) And although this part of the market may not be the bulk of the token volume, it is the bulk of the market value.
That's because a lot of human knowledge work is too generalized and fuzzy for dedicated, fine-tuned models, so they are almost entirely different markets that don't particularly compete with each other. (Though if SaaS companies successfully build around verticals that can use small models applied against well-defined jobs, there may be opportunity to push the small/big capability boundary to subsume marginally more valuable tasks that today would require mid-grade reasoning.)
This can change quickly though, so it's not that big of a deal. If AI energy demands push the Gov to deregulate/fast track new plants, or the industry decides to build out their own generation renewables.
Across sectors, China added 543 GW of energy in 2025. Next year, USA is expected to add between 70 and 80 GW of energy
Which is to say, this isn't really a lock-in/ stickiness vector (unless maybe the wording itself of a skill is hyper-optimized for a specific model)
However with the latest models Fable, Kimi K3, 5.6, it's getting to a point where I sometimes forget what model I am on without noticing a difference. And once I realize it because something may not be exactly like I expected it I won't switch for that work either because I don't want to invalidate the cache.
For the next work I will do there is maybe a 50/50 chance to remember to switch the model before I start.
That's not what I would call stickiness towards a certain provider.
What is the cost of AI? The single largest ingredient is Nvidia profit margin.
Huawei accelerators are not as efficiency yet, but they don’t nearly extract as much margin.
Why would future revenue stay with the labs given this situation? This whole thing had an airline industry sized red flag on it that makes investing into frontier lab about as sexy as investing in United.
Maybe the token economy is some kind of reverberation of the airline reward miles economy, the emergency hatch to be able to survive under maximal supplier extraction (Nvidia is just the top of a monopoly stack here, even if they replace those chips, the HBM, ASML, Foundry layer can get their dues)
Sorta yes, sorta no.
A single 5090 consumes 450W - at Californian energy prices of $0.38 per kWh that's $0.17 per hour. And the card itself costs $4100 on amazon. So after 2.75 years running at full power 24/7 you'll have spent more on electricity than on the card. I would have thought most data centres being built today would have a design life longer than 3 years.
Of course you can throttle the cards to ~300W without losing too much performance. But also you need more than a single 24GB card to run most modern LLMs.
> 132kW
> ~$3.7-4M
So about 300x the 5090's power but 1000x the price. Roughly 9 years for electricity to exceed price at $0.38 and datacenters will show up in areas with cheaper power than CA.
_aavaa_•17h ago
Sounds great to me; live by the sword, die by the sword.
ronsor•10h ago
Sorry, OpenAI & Anthropic.
noncoml•9h ago
> what are large language models but the distillation of all of the knowledge on the open Internet, scraped by the frontier labs and distilled into the models
If it’s as easy as that why do they choose to distill another model and not distill the knowledge on the open Internet from scratch?
numpad0•9h ago
root_axis•8h ago
paxys•8h ago
A model trained on all knowledge from the internet (and other sources) is large but ultimately not very useful by itself, because it is going to spit out all kinds of garbage. You have to apply multiple further stages of training and refinement to the base model before putting it in front of users. So as an example you can train a model by yourself and then have GPT or Claude continuously check its outputs and correct it when it is wrong, ending up with a far more powerful model.
eli•9h ago
But barring the terms of service from forbidding distillation seems like a tough sell. OpenAI shouldn't be allowed to decide what types of customers it wants and doesn't want?
mediaman•9h ago
eli•7h ago
The terms of service don't even necessarily matter here. OpenAI could cancel your account for almost any reason, or for no reason at all. They don't particularly need to cite a ToS violation just as a store owner doesn't need to point to a written policy to kick you out of their store.
If the underlying issue is that LLMs should be regulated as a public good, then lets have that discussion. If it's that the major AI companies are becoming too powerful and anti-competitive, let's talk serious anti-trust enforcement. Micro-managing business policies isn't going to work very well.
hnfong•2h ago
You are pointing out that OpenAI can cancel user's accounts for almost any reason, and nobody can really force them to serve customers that they suspect are distilling their models.
That's one thing.
The GP is saying the government can make laws to make terms against distillation unenforceable. Without such laws, if you signed an agreement with OpenAI pinky swearing you won't distill, but turns out you did, you are liable in tort and OpenAI can sue you. (It seems nobody really cares about contract and agreements any more, but still...)
This is the other thing.
And I think you are both right.
llm_nerd•9h ago
It's also a bit of securities defensiveness. Pretending that you really do have a super moat, people just keep swimming in it so you just need to add more alligators.
It's farcical. Anyone who has worked on large models knows that the premise that an almost-Fable model was trained with distillation is beyond ridiculous. It's theoretically possible if they spent tens of billions of dollars on API calls, but it isn't the magic that somehow these people keep convincing people it is.
Previously Anthropic has reported on some Chinese firms doing chicken-shit level of API calls, that at most would be doing some Q and A or final fine tuning. The notion that they're training these models via it is fantastically ignorant nonsense that only very ill-informed and gullible people fall for.
ultrablack•8h ago
llm_nerd•8h ago
This isn't the big gotcha some people seem to think it is, and the whole news cycle about that was mostly by people who have no idea what they're talking about. It's actually a meaningless data point. But it's precisely the sorts of people who think that a few thousand free accounts surreptitiously snuck off with Fable.
qurren•9h ago
ToS is just conditions that you agree to in order to use a private service that is provided at-will. I can have a private coffee shop where the terms of service are that you must wear red to enter, and if you're not wearing red, you are not welcome on my property.
So it would be upto OpenAI and Anthropic to enforce them on their own terms (by banning accounts and IPs).
ascorbic•8h ago
nl•8h ago
onesociety2022•8h ago
ButlerianJihad•7h ago
grim_io•8h ago
chuckadams•8h ago
scotty79•8h ago
thesmtsolver2•8h ago
We are discussing Chinese models. Now look at how much foreign competition the Chinese government prevents in their domestic market in other industries.
scotty79•7h ago
matheusmoreira•
bluegatty•8h ago
Distillation is just value extract.
It's soft, and I'm not sure what the answer should be ... but I think that there is a difference.
I think we start by recognizing that ... and then try to figure it out from there.
'The Internet' may be a public good, maybe we make them pay a tax for that, but that's different than distillation.
scotty79•8h ago
There is a value-add in selecting the valuable parts out of the garbage. And let's face it. Largest models contain a lot of garbage.
bluegatty•7h ago
We ought to identify that and integrate that into our thinking.
nemomarx•7h ago
bluegatty•7h ago
Literally the biggest thing of our generation - AI - is the living embodiment of that 'value add' writ large.
'What is the difference' - is the AI you use all day, in comparison to 'all the world's data' you can use for stuff and do 'whatever' with it, but are not likely to come up with something hugely useful otherwise. Maybe, not likely, if you did, it would be 'value add'.
cayley_graph•8h ago
matheusmoreira•6h ago
Felony contempt of business model.
magarnicle•6h ago
What I'm saying is, doesn't the law already cover 1?
_aavaa_•5h ago
In the US one of the factors is “ the effect of the use upon the potential market for or value of the copyrighted work”.
If anthropic Hoovers up the world’s books and trains on them, and then spits them out verbatim on command, then it will clearly impact the value of the work; nobody will buy the original, they’ll just ask Claude.
Others also argue that even if it’s not reproducing it exactly that the training runs afoul of that factor, specifically the “market for” portion. A rights holder can no longer license their book for training of LLMs if Anthropic goes ahead and just trains on it anyway.
magarnicle•4h ago
Ah, right. So if we want models to be capable we need them to be trained on as much as possible, yet we also want to stop what you described. So what can be done?