So from my perspective, it's doubtful that this is the moat. Besides, for example, Claude Code in particular is so buggy (and always has been).
He talks about this in another recent essay https://stratechery.com/2026/anthropics-safety-superpower/
> If you own the user touchpoint, then you have meaningful lock-in, and the best way to own the user touchpoint is to be the canvas for everything they need to do. This, by extension, means that the frontier labs are on a collision course with software companies: it’s software that owns the user touchpoint, and it’s in the frontier labs’ long-term interest to not simply be a commodity input into software but to simply replace software outright.
I'm amazed that no one is talking about proposals that are surely being discussed in Washington and pushed by SV lobbyists to restrict Chinese models on national security grounds, or other some other basis.
The belief that Bytedance could engineer a finger on the algorithmic scales to serve the interests of the Chinese Communist Party led to a lot of debate in Washington, and ultimately resulted in TikTok being divested from its Chinese owners. Huawei is shut out from the U.S. market, which limits its business even in markets where it's not banned because it's effectively stamped with a scarlet letter.
IMHO, Chinese models are headed for a similar fate or at least a showdown in Washington or the courts because they are supported and/or controlled by entities which ultimately serve the CCP.
Personally, I think models will increasingly become specialized in different areas, some good at X, others good at Y, and we might see workflows that mix multiple models.
"I am personally surprised the Chinese state continues to allow the open sourcing of models this good, given potential risks" what risks?
I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). Confused what this means
Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. Confused again.
One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product, AI is a "public good" which will ultimately be provided by the state as a kind of "digital public infrastructure." This future strikes me as a dystopian hellscape, but I've never met an open-weight models advocate who doesn't ultimately concede this is where things end. I don't understand this at all.
Can someone in the know please use plain layman's terms to explain what this tweet is about?
how is running servers supposed to be 0 cost, while running ai inferrence isn't?
A single response from kimi k3 requires hardware that cost between 500k and 1m dollars up front and draw over 20kW. Each request costs at least 5% to 10% of the charged cost.
My experience has been quite the opposite. I was using Claude Code almost exclusively this winter/spring and swapped to Codex earlier this summer. It took no time whatsoever to switch. And before Claude Code, I was using Cursor. Same story.
[edit: Oh and there was also a brief interlude with Conductor, though I think they're more or less just serving the underlying Claude/Codex harness]
Probably the only reasons I would seek change are economical.
This has an element of stochastic improvement so it's hard to predict but the chance of the U.S. "winning" this "race" is pretty bleak.
You see this all the time in communities that have internalized hierarchy as a "good", little kings of shit mountain vying for less and less at a higher and higher cost.
An astute Chinese analyst could reasonably forecast that they had little chance of controlling the AI market due to sovereign trust issues, but would also note that AIs are just software.
When the dust settles the US still won't have factories, and the real value of AI models is still going to be embodying them and getting them to do real, consumer facing work.
Perhaps the most striking thing about the AI boom is how quickly the US abandoned the veneer of local manufacturing in favor of more expensive buildings producing nothing you couldn't make anywhere else on the planet...from imported parts.
China has a billion+ people that their AI can "study". Plus due to China's political structure, their AI has access to everyone's chats, comments and sites, scraping everyting.
Here in the US, with 1/3 the population, the AI race was lost before it even began. Plus in the US, all companies and people are doing all they can to restrict AI from scraping sites and peoples chats.
So I believe, China will end up owing AI.
I think you hit the nail on the head - right there!
for VCs, breaking even is losing
They’re going to try their best to offload these investments into our pensions before the inevitable crash.
https://finance.yahoo.com/markets/stocks/articles/goldman-sa...
> open weights, open code and open data
Even if you have all these things you still can't replicate a model because of randomness.
You can backdoor a model with less than 1000 examples and it is impossible to detect.
There are also half a dozen other companies from China continuously hammering our clients’ websites.
I was wondering, what's in that cold dessert? Low and behold satellite imaging shows massive datacenter build outs, very cheap solar energy.
Few months ago something happened and the Geo location on data on those IP now shows "Shanghai" or "Shenzhen". A way to cover tracks? But mapping latency still points to fact that nodes behind these IPs are still operating around Xinjaing region
credit:
'You Can't Cheat Time: Finding foes and yourself with latency trilateration' https://youtu.be/_iAffzWxexA HN user: lopoc
Shenzhen vs Xinxiang is hard to do using this technique but Shanghai vs Xinxiang does show difference.
Assuming that China only distills is a huge mistake.
It’s no longer some backward place that does low value copying. Look at companies like ByteDance and Xiaomi.
Chinese companies aren’t just distilling, they’re acquiring data in the same way American companies did by paying people and crawling the internet.
The way I understand it, China has a few large companies that crawl the web at a rapid rate and build corpora. The government essentially wants select few companies to do this and then make the data available to other strategic companies operating within China.
Then there are data aggregators that buy data from apps, websites, and services, as well as systems like OpenRouter or Cursor, where companies can learn from the “traces” of coding agents, chats, and so on.
This massively reduces costs, as smaller companies like DeepSeek don’t have to do their own crawling or acquire data from 100s of websites and coding agents etc....
There are also companies in China that buy American LLM APIs and proxy them to companies within China. So, there could be 10,000+ companies using American AI products, while China logs all of this, understands how they’re being used, and trains on their traces.
Sure as an indie hacker, you could go download the weights for a Chinese model with a VPN, and then attempt to run it at home by building your own GPU cluster but these large models require quite expensive hardware to run on and so it makes it less likely than anyone would invest that much capital to do something that is illegal. There's no way for them to sell a legal service using those tokens. So it can only be strictly for personal use (the Govt won't care because very few people will have that kind of money and risk appetite). The other option will be that there will be some shady third-party providers in foreign countries who are willing to sell tokens from these models to US consumers knowingly.
I think we see this with Meta being paranoid about internal Claude usage, to avoid inadvertently distilling[1].
If distillation is a driver, then smaller American labs could be distilling, but are not for legal reasons.
But that's a big if we just don't know for sure.
1 - https://cryptobriefing.com/meta-restricts-claude-code-codex-...
This from OpenAi's Head of Strategic Futures "Some observations on Kimi: It's a very good model! I don't think its performance can be explained away by distillation or anything like that"
https://x.com/deanwball/status/2078133895766114412
China's strategy of spending billions on training these models and open sourcing these models away is strategic - they want to kill the US LLM industry at any cost.
To win on the AI front by any means necessary.
Why wouldn't it be? China is pumping out AI research and researchers at a staggering pace and there is no inherent reason why western models should be better
Why is it when Anthropic and OpenAI spend billions trying to beat each other it is competition, but when the Chinese companies do it then it is trying to kill the US LLM industry at any cost.
The US federal government spends billions in subsidies via the US Chip Act, and bans chip sales to China to support US companies.
But the implication is that somehow Chinese competition is illegitimate because "strategic".
I see massive risks in belief the inferences drawn from strategic information cannot be seen. So if you depend on some position remaining inside a secure facility but you drove to it from data outside that secure facilty, The likelihood that an inference model can derive the same idea is very high. Collation over public data is not inherently secret because you used a secret model or secret weights.
A more simplistic take might be that the fear is not actually driven in the secrets, the fear is "the emperor has no clothes"
Is this an assertion that is backed by evidence?
From the Elon/OpenAI trial:
> On the stand in a California federal court on Thursday, Elon Musk was asked if xAI has used distillation techniques on OpenAI models to train Grok, and he asserted it was a general practice among AI companies. Asked if that meant “yes,” he said, “Partly.”
https://techcrunch.com/2026/04/30/elon-musk-testifies-that-x...
The lessons from steel, solar and EV needs to be learned by all lawmakers. You have to respect and learn from how China Government puts the system in place for complete industry takeover and they have been very good at it. The problem with AI is that democracies will be inherently slow in adopting AI, unless something changes in the system.
At minimum, every democratic Government (US, Europe, India) need to build long-term AI vision and execute that no matter which party comes to power. Additionally, be ruthless about protecting domestic labs. It can only be possible if the intelligence pricing by domestic labs per productive task is in the similar range as open-weights models. Right now, it is not the case, even if the article gives the example of Sol vs K3.
Protecting domestic labs means not bailout, but fast track to cheapest energy, fast track approval for data centers, enforce some guardrails so customers get to use the open weights models only hosted in the country by US (or Europe) businesses. Without these protections, it might be a slow death.
It doesn't have anything to do with the form of government, it has to do with the aims of the government.
lmao
That's a big claim that his whole thesis rests on but is largely not backed up. Where are the apples-to-apples tokens-to-answer benchmarks that he's using - doesn't look like there are any, just a handwavy implication that US models are more token efficient. And US labs may be in much different situations from one another: it's known that some labs like OpenAI bought big, early on compute and may have secured better pricing.
His article also does not mention the average price of electricity in China vs the US, which it seems like China leads on, and probably has the political power to more heavily subsidize. While I agree the COGS is often overlooked by top line benchmarks on coding tasks, etc, it seems that he's running on a big assumption while claiming "labs on the frontier will be fine".
I have sonnet do the thinking, deepseek does all the tasks. I've massively reduced costs with this approach.
I think it is the right move to protect American interests
This was my assumption as well. It's also generally true of 'traditional' deep learning models that inference cost is expensive compared to training.
But the cost per token for inference has been very quickly dropping. I don't recall where, but I recall about ~50x down from GPT3, even as model complexity has increased. Even with agentic systems, there are lots of optimization opportunities. I'm less assured about claims like this.
I have been working on a project with about a dozen generation tasks, each of which comes with a fixed token budget. The nature of this system requires that most tasks be completed by distinct model families.
As a result, I tested ~50 models across as many model families as I could gather, frontier and open weight, API (gateway and direct) and self-hosted. Evaluation was based on a set of cosine similarity validations that was repeated across ~50 different embedding models.
Interestingly, frontier models did worse on the tasks than open weight models. However, when it came to costs, the picture was reversed: frontier models were much, much more token-efficient. In fact, almost no open-weight model was able to meet the initial token budget, while almost all frontier models did. Moreover, open weight models struggled massively with reasoning, in terms of latency and token consumption.
I also found that the latest models did not perform better than older models. And any a priori benchmarking data was utterly useless.
So, I ended up using a set of open weight models without reasoning, as it turned out reasoning as well as frontier negatively correlated with the tasks. However, before I knew this, I had spent a lot of time running each available reasoning level for each model.
Lastly, as an aside, when it came to embedding models, size (dims as well as model size) did not correlate with quality, once a hurdle figure (~2k dims) was met. In fact, sweet spot was 3-5K, and for my (text-based) set of tasks, dense models tended to outperform MoE ones.
Is he casually assuming a singularity has already happened? A regular first-mover advantage I can understand, but those have been squandered or lost many times before.
Sure, any model that is not at the frontier can use the frontier model to generate synthetic high quality training data, so this can reduce significantly the training costs.
But at the scale of OpenAI, Anthropic and Google, it is quite likely that the (raw) training cost is very high anymore. Here's a few heuristics:
1. All the hyperscalers see a huge demand for inference. They can't deploy datacenters quickly enough to satiate all the demand they see. But, it's is impossible for the inference demand to be constant throughout a day or a week. If you use the times when the demand is lower than the peak demand (which is almost all the time) to dedicate the spare compute capacity to training, then your the cost of training compute is zero.
2. It is likely that increasingly a higher cost of the "training" is actually setting the guardrails, which is essentially post-training. As we've seen, without proper guardrails, the US Government won't allow you to serve inference. Anthropic was hit directly, but OpenAI delayed their 5.6 release as well to make sure the US Government is ok. This part of the training cost can't be reduced easily by using synthetic data generated by other models.
3. The frontier labs are also investing more and more in building an ecosystem around their models.
I am not a frontier lab insider, but take a look at the jobs posted on the Anthropic career page [1]. There are 74 jobs in "AI Research and Engineering" and by my count at most 15-20 are related to pure model training (of pre-training or RL type), and the rest are post-training, safety and security, alignment, interpretability, productivity and lots and lots of other things.
The actual difference is how much scrutiny and time was put into the Mythos / Fable and GPT 5.6 release. Making it feel like “these are a big deal”. Spring and summer THAT was the AI story
Then Chinese labs release models that approach Fable performance. We’re shocked they just seemed to appear out of nowhere.
It’s less about the gap closing. It’s more about the weight we put into Fable-capable models.
"You're trying to kidnap what I've rightfully stolen!" -- Vizzini
Correct. It shouldn't be allowed to do that.
https://m.economictimes.com/industry/renewables/china-wto-co...
Copying something is not.
Programming Microsoft Word is value add, copying the code is not.
Copying design ... there are some question marks there.
It's extremely easy to understand at it's core.
What makes it hard, is that faux intellectuals like to deconstruct ideas at the margins, and have those critiques stand in for reason.
"At sunrise the sun is only 'half there' ... there fore there is no 'day and night' just a blur! Day and night are the same thing!"
The training data used is part of all of this is a separate but related question.
For this reason alone I would also argue that the idea about an agent harness being sticky is a non-starter long-term.
I think it's referring to the belief that LLMs are not the path towards AGI, and that LLM's, while useful, are not going to have the impact that the American labs believe it will have.
The Silicon Valley people like this openai guy, high on their own supply, are convinced they are building some machine god that will either bring about the end of the human race or utopia, they therefore cannot understand why the Chinese (or any other normal person on earth) are not afraid of chatbots and have other things on their minds.
I assume they mean the risk of opening up "forbidden" knowledge to the masses without adequate control, which the CCP hasn't historically been known to do.
> I suspect the reason they are is 75% explained by strategic blindness/lack of AGI-pilledness (the CCP is very Yann Lecun-y in its views of AI). Confused what this means
Yann Lecun is a pioneer in the field of AI and Meta's former AI head. He is famously anti-LLM, and considers the entire technology a dead end to achieving human-level AI. The author is saying the CCP has similar views (that LLMs aren't going to get exponentially better/lead to AGI) which is leading them to not control these models as tightly as they otherwise would.
> Open-weight models are inherently decelerationist, and I'm continually surprised to see the so-called "accelerationists" so excited about open-weight models. Confused again.
"AI accelerationists" = people who want AI to progress. According to the author these people should not celebrate open models because open source = less commerical value in LLMs = less investment into the field (because how are companies going to get returns?), and this will ultimately lead to slower growth.
The last bit is about government controlling AI vs commercial companies. According to the author the former is a dystopian hellscape.
IMO even if you think his points make sense, his job title ("head of strategic futures @openai") means they should all be taken with a massive grain of salt.
I’m struggling to understand this perspective. Is he using the words accelerationist/decelerationist in a sense other than the obvious one?
EDIT: I searched his twitter history and discovered that his argument is basically “if you drive down costs, then OpenAI will have less money to invest in development, slowing down the overall rate of AI progress.” IMO this take betrays an overwhelmingly stupid degree of exceptionalism, but I guess that’s what I’d expect from someone working at OpenAI.
For companies, these decisions are very sticky. Companies go through a lot of red tape to get anything purchased and approved, then they discourage change because it's a lot of work.
So the product that gets a foothold in a company sticks for a long time.
Then a couple years later a sales person convinces an exec that they can save some money by switching, so the switching game begins. Not necessarily motivated by the better product, mostly the price. My wife's company keeps switching their tools out from under everyone every year or two. Just when they get everything stabilized and everyone familiar with the new tool, some new contract is signed that moves them all to some other company's suite.
You install MCP connectors, specific skills, work around model/harness quirks, set security boundaries etc.
It's a lot of work, and most people will never want to change it once they have it working.
its so distracting seeing these types of confision.
every plugin is already just multimodaling their targets.
The cost of me moving around these different AI models and harnesses was pretty much 0.
- Commodity market profitability is determined by marginal cost of production. LLMs have marginal cost; traditional software does not.
- Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use
- The highest tier Chinese models are not more economical than US frontier models. Try GLM 5.2 and see how much it costs to do real work. I did, and it was more expensive than GPT 5.6.
- This is because US labs are leading on cost efficacy of inference ($/task)
- Training will decline as a percentage of costs as inference expands compute share due to agentic workloads. A big part of training now is optimizing token efficiency. It's hard to distill token efficiency; that is perhaps why Chinese LLMs are so inefficient.
- With increasing inference as % of total compute, if labs create efficient models -- which they can, because they can create highly optimized models amortized over very high inference loads -- they can be low cost producers, and be competitive at $/task rates
I appreciate the transparency in explicitly stating their motivation for writing the article (a response to what the author saw as an overreaction to Chinese models), but I feel the article goes too far the other way with multiple unsupported leaps of logic, and overstating the stickiness of the products.
> I highly doubt that Chinese models are cheaper to serve on a marginal cost basis, they just seem cheaper because Anthropic and OpenAI are so supply constrained that they are charging far more than they would if there were sufficient supply to meet the demand for intelligence. [emphasis mine]
I guess I'm missing the part of this article where they bring hard numbers in to back up the argument here. What work was attempted? https://cursor.com/evals shows the previous generation of open models (Kimi K2.7) trading blows with the others, cost effectively. Composer 2.5 is itself a fine-tune of K2.7, and it's apparently quite token efficient, so why would it be impossible for a Chinese lab to achieve something similar? GLM 5.2 Max is also ranked above the lower end OpenAI models and is not far off in price.
It's weird to have this entire discussion about tokenomics without mention of the circular financing and debt raised by labs in the West, which can then essentially give away their capacity to end users. OpenAI giving away quota resets to subscribers like candy on Halloween while their compute partner Oracle's bonds is reevaluated to be one grade above junk? How?
I don't think you can make an argument about the future one way or another by arguing using the listed prices. The math is not internally consistent enough for it.
Because you're comparing retail price whereas the parent commenter (and the article) is talking about marginal (ie. inference) costs. American labs are providing a premium product and they're charging accordingly. Meanwhile for chinese models they're open weight so they're limited to how much they can charge without competitors undercutting them.
If we use tokens as a rough proxy of inference costs (rough approximation, I know) and look at artifical analysis benchmarks, you see that all the open models are behind the pareto frontier in terms of efficiency.
Anthropic’s API pricing is getting impossible to justify. Anthropic previously had the highest quality models, and used their position to charge premium prices, enjoying inference margins of over 70% [0]. They could charge these prices because no other model came close.
But over the past month, the market has shifted dramatically. Over every single performance tier, Anthropic is being squeezed on price.
* Low end: DeepSeek V4 Flash runs at ($0.02/task), Xiaomi's MiMo-V2.5-Pro at ($0.03), and Haiku at ($0.24). Anthropic is ~10x more expensive than the Chinese open-weight options.
* Mid tier: Claude Sonnet 5 ($1.53/task) is nearly 50% more expensive than GPT-5.6 Sol ($1.04), nearly 2x the cost of GPT-5.6 Terra ($0.82), and 3x the cost of GLM-5.2 Max ($0.47). There is basically no reason to ever use Sonnet 5, the competitors are significantly cheaper.
* High end: Opus 4.8 ($1.80/task) and Fable 5 ($2.75) are the two most expensive models, and GPT-5.6 Sol ($1.04) and Kimi K3 ($0.95) offer comparable performance for significantly less. Less the fact that Kimi K3 will get ~10x cheaper once its weights are released and served on neoclouds with Nvidia hardware [1].
OpenAI priced their latest GPT-5.6 models cheaply in order to regain market share. When Anthropic clearly had the best models, their 70%+ inference margins were defensible. But today they are the most expensive option in every single tier. Unless they make significant price cuts soon, they run a serious risk of bleeding market share.
[0] https://www.mindstudio.ai/blog/anthropic-inference-margins-7...
[1] "American companies such as Modal, Fireworks, and Baseten will be able to serve Kimi K3, at one-tenth the cost of their Chinese competitors because they have access to advanced Nvidia hardware" https://x.com/rohanpaul_ai/status/2079027313455550839
This is a flatly false statement for most things powering backend applications. The AI consumer "doing real work" model, either for analysis, chat, or coding could well be more cost effective with closed frontier models.
But most of these internal glue business SaaS applications where engineers are integrating are not those tasks. It is those tasks which 1) drive immense amount of domain-specific data into the platform over time, and 2) are most encouraging of driving open model independence with no vendor lock-in.
Anyone on this site who has actually used ML models (more accurate in many cases) knows there's a lot of kludge that simply does not need a 5 minute agentic feedback loop to solve the problem. And they were solvable a year ago with lower class models. The token economics are exceptional and the anecdotes of a16z saying 80% of startups are productionizing open models is only surprising to people who think running your company on OracleDB in 2026 is a sound engineering decision.
It may not be false but may be a "category error" [0]. Reserved GPU pricing & bulk inference pricing is 3x to 6x cheaper than "API rates", but renting your own GPU cluster (in this crunch) to run a 600b+ open weights is going to be "more expensive than GPT 5.6".
Even then, it remains to be seen if Huawei will pull their weight (and match up to Nvidia) as spectacularly as their fellow Chinese AI Labs have. If so, the WAICO alliance is ready to go all-in.
[0] Ben, and probably other "influencers" in this space, may be prone (knowingly or unknowingly) to favour points that make their conclusion for them (https://en.wikipedia.org/wiki/Motivated_reasoning).
> Models are not free. Downloading them is free. Running them is not. This has manufacturing economics, not software economics; the idea that they are "free" is an economic category error as it relates to their actual use
I think a large part of manufacturing economics is illiquid overhead and the cost of expertise to set up and run your manufacturing line. Compute economics don’t have the same illiquidity nor do they require the same expertise or even specialized infra (current temporary chip shortage aside).
The implications of this are small players (e.g. your uncle running an inference server out of his garage) have comparably efficient marginal costs as big players. Compare this to actual manufacturing where small players have essentially no access to the manufacturing facilities of the big players.
Additionally, big players with a lot of compute who are not meaningfully in inference today (e.g. Amazon) have a fairly straightforward glide path to utilizing that compute to compete.
> This is because US labs are leading on cost efficacy of inference ($/task)
It’s possible, but I would need to see better data on this.
>A big part of training now is optimizing token efficiency. It's hard to distill token efficiency; that is perhaps why Chinese LLMs are so inefficient.
I think it’s fair to assume this is true, but also token efficiency is not a meaningful competitive moat. It’s not like these are secrets the Chinese will never figure out, it’s a fairly active research space and the outcomes are quantifiable.
Its the user base (with ads and upselling) and proprietary wrappers which will make money for typical customer.
Even enterprise customers arent going to be spending a lot on tokens. Once labs no longer have to subsidize trainings tokens costs will drop 10x and once models get burned on chips costs will drop 10x more and you physically won't be able to burn significant number of tokens unless you're deliberately trying to.
Correct. These chinese labs has proven that having just the model is not a moat, and the safety concerns were all just attempts at regulatory capture.
This is why labs like OpenAI and Anthropic are panicking and are racing to the exit before their valuations start being questioned.
Even with this, the cost of verification would be enormous. You would need a massive cluster to repeat the training E2E.
But if we have to look at what we think margins might look like, DeepSeek continues to host v4 Flash at the existing price despite competitors beating it in price (https://openrouter.ai/deepseek/deepseek-v4-flash), so there's at least one example of a Chinese lab charging a predetermined price despite competition. And no one but Moonshot is hosting Kimi K3 yet (https://openrouter.ai/moonshotai/kimi-k3). Perhaps there's room in the market for those who release their models to make margin on them.
And I believe my Composer example speaks for itself. The open models are behind but there's tangible proof they can be tuned for pareto frontier efficiency. See "Cost per Task" at https://artificialanalysis.ai/agents/coding-agents.
their competitors are discounted at around 33%, so it's safe to say that's the margin, maybe less if their competitors have worse caching or quantization. Meanwhile claude code/codex resellers selling tokens for 90% off API price, presumably by reselling usage from fixed consumption plans, which gives an idea on how fat the american labs' margins are.
>And I believe my Composer example speaks for itself. The open models are behind but there's tangible proof they can be tuned for pareto frontier efficiency. See "Cost per Task" at https://artificialanalysis.ai/agents/coding-agents.
But composer is a closed model? If it's really that easy to get better coding performance, why haven't the chinese labs replicated it? And this is all assuming the performance boost is real and not from benchmaxxing. Moreover if you apply the "street price" discount I mentioned above, American labs look far more favorable.
I look at that and think that they must be losing money hand over fist on something like this, not that this shows what their margins are like. If their margins are like this then I don't see why they'd be raising money and shuffling it around in circles.
> If it's really that easy to get better coding performance, why haven't the chinese labs replicated it?
Nobody said it would be easy! I just think it's possible, and that presumably they will get around to doing it at some point.
But much like Ben's point that commoditization is a relatively novel concept to many in tech, it's not the consumer AI applications at risk of commoditization. They have distribution there.
It's the literally millions of engineers who are updating codebases with tools replacing workers partially or wholly. It's the supply-side where there's compression, and no need for distribution.
I would argue, given the enormity of the existing SaaS stack and how it integrates with the human machinery of personnel, that's where volume is. And that is clearly cheaper and a home run.
Commoditizing a ~$100B AI consumer market is no small feat. Commoditizing 20% of the $500B SaaS market, to say nothing of the underlying systems in the who-knows-how-many trillions "Big Tech" market (you're obligated to say that like the Kool Aid man), is shocking.
The labs are not interested in the small, fast, single purpose end of the market. Google increased their pricing on Flash so much that it stopped becoming a cheap model; instead, they released Gemma 4 open source, which is actually easier to use from a third-party inference provider than from Google.
From a total token volume perspective, these "utility" models (classifiers, simple summarizers, small OCR models) will absolutely drive enormous volumes of tokens, at low prices and margin and modest overall market size. Because the models are small and the performance requirements are modest, and because their use cases are specialized rather than general, there are poor economies of scale: they can run cost effectively on rented small GPUs, and a big player doesn't get a structural cost advantage. These models are usually 1b - 30b in size, and can run on a rented 5090. I've productized these myself: I run millions of pages through a fine tuned 1b OCR language model that runs on 5090s at a cost far lower than commercial providers.
But that's not the segment of the market where GLM 5.2, Kimi 3, etc., play. They compete with frontier capabilities, and they are not particularly cheaper than OpenAI models at a cost per task. (I do actually think they compete well with Anthropic, because Anthropic's model efficiencies are poor compared to OpenAI.) And although this part of the market may not be the bulk of the token volume, it is the bulk of the market value.
That's because a lot of human knowledge work is too generalized and fuzzy for dedicated, fine-tuned models, so they are almost entirely different markets that don't particularly compete with each other. (Though if SaaS companies successfully build around verticals that can use small models applied against well-defined jobs, there may be opportunity to push the small/big capability boundary to subsume marginally more valuable tasks that today would require mid-grade reasoning.)
_aavaa_•11h ago
Sounds great to me; live by the sword, die by the sword.
ronsor•4h ago
Sorry, OpenAI & Anthropic.
noncoml•4h ago
> what are large language models but the distillation of all of the knowledge on the open Internet, scraped by the frontier labs and distilled into the models
If it’s as easy as that why do they choose to distill another model and not distill the knowledge on the open Internet from scratch?
numpad0•4h ago
root_axis•2h ago
paxys•2h ago
A model trained on all knowledge from the internet (and other sources) is large but ultimately not very useful by itself, because it is going to spit out all kinds of garbage. You have to apply multiple further stages of training and refinement to the base model before putting it in front of users. So as an example you can train a model by yourself and then have GPT or Claude continuously check its outputs and correct it when it is wrong, ending up with a far more powerful model.
eli•4h ago
But barring the terms of service from forbidding distillation seems like a tough sell. OpenAI shouldn't be allowed to decide what types of customers it wants and doesn't want?
mediaman•3h ago
eli•2h ago
The terms of service don't even necessarily matter here. OpenAI could cancel your account for almost any reason, or for no reason at all. They don't particularly need to cite a ToS violation just as a store owner doesn't need to point to a written policy to kick you out of their store.
If the underlying issue is that LLMs should be regulated as a public good, then lets have that discussion. If it's that the major AI companies are becoming too powerful and anti-competitive, let's talk serious anti-trust enforcement. Micro-managing business policies isn't going to work very well.
paxys•2h ago
llm_nerd•3h ago
It's also a bit of securities defensiveness. Pretending that you really do have a super moat, people just keep swimming in it so you just need to add more alligators.
It's farcical. Anyone who has worked on large models knows that the premise that an almost-Fable model was trained with distillation is beyond ridiculous. It's theoretically possible if they spent tens of billions of dollars on API calls, but it isn't the magic that somehow these people keep convincing people it is.
Previously Anthropic has reported on some Chinese firms doing chicken-shit level of API calls, that at most would be doing some Q and A or final fine tuning. The notion that they're training these models via it is fantastically ignorant nonsense that only very ill-informed and gullible people fall for.
ultrablack•3h ago
llm_nerd•3h ago
This isn't the big gotcha some people seem to think it is, and the whole news cycle about that was mostly by people who have no idea what they're talking about. It's actually a meaningless data point. But it's precisely the sorts of people who think that a few thousand free accounts surreptitiously snuck off with Fable.
qurren•3h ago
ToS is just conditions that you agree to in order to use a private service that is provided at-will. I can have a private coffee shop where the terms of service are that you must wear red to enter, and if you're not wearing red, you are not welcome on my property.
So it would be upto OpenAI and Anthropic to enforce them on their own terms (by banning accounts and IPs).
ascorbic•3h ago
nl•2h ago
onesociety2022•2h ago
ButlerianJihad•2h ago
grim_io•3h ago
chuckadams•2h ago
scotty79•2h ago
thesmtsolver2•2h ago
We are discussing Chinese models. Now look at how much foreign competition the Chinese government prevents in their domestic market in other industries.
scotty79•2h ago
matheusmoreira•
bluegatty•2h ago
Distillation is just value extract.
It's soft, and I'm not sure what the answer should be ... but I think that there is a difference.
I think we start by recognizing that ... and then try to figure it out from there.
'The Internet' may be a public good, maybe we make them pay a tax for that, but that's different than distillation.
scotty79•2h ago
There is a value-add in selecting the valuable parts out of the garbage. And let's face it. Largest models contain a lot of garbage.
bluegatty•2h ago
We ought to identify that and integrate that into our thinking.
nemomarx•2h ago
bluegatty•2h ago
Literally the biggest thing of our generation - AI - is the living embodiment of that 'value add' writ large.
'What is the difference' - is the AI you use all day, in comparison to 'all the world's data' you can use for stuff and do 'whatever' with it, but are not likely to come up with something hugely useful otherwise. Maybe, not likely, if you did, it would be 'value add'.
cayley_graph•2h ago
matheusmoreira•1h ago
Felony contempt of business model.
magarnicle•31m ago
What I'm saying is, doesn't the law already cover 1?
_aavaa_•21m ago
In the US one of the factors is “ the effect of the use upon the potential market for or value of the copyrighted work”.
If anthropic Hoovers up the world’s books and trains on them, and then spits them out verbatim on command, then it will clearly impact the value of the work; nobody will buy the original, they’ll just ask Claude.
Others also argue that even if it’s not reproducing it exactly that the training runs afoul of that factor, specifically the “market for” portion. A rights holder can no longer license their book for training of LLMs if Anthropic goes ahead and just trains on it anyway.