Edit: Apt domain.
And the only data they are showing is that cache prices went down for new Claude/OpenAI models but that's proving nothing, IMO.
If local LLMs get "good" enough, people will soon paying for subscriptions to ChatGPT and Claude, which hurts their revenue.
Think you missed a word there.
Given the recent deepseekv4.1 advances - how good of a 3B model can we make to run on an iphone natively? is it good enough to match common muse/dot use cases for consumers? the phone is already always on.. no need for a cloud server.
It is vanishingly rare I ask an older model to do any task. Newer bigger and smarter models will just do the task better.
Therefore, I believe we are nowhere near 'good enough'.
I never drive my steam engine to work these days. It isn't good enough.
Does anyone know if there are any distillation datasets available? I'd love to see these distributed on BitTorrent. I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
It’s sort of like gun nuts arguing that more guns is the answer. I mean, ok, maybe you’re a responsible gun owner or AI user but relying on personal responsibility doesn’t fix systemic problems. There are bad people out there.
They do claim that it violates their ToS, which we can assume is simply correct, since they get to put whatever they want in their ToS.
Given all that, I don't know what the fuss is. Are they supposed to not use the advances that were openly published by Chinese labs? The entire industry is built on a discovery made at Google, which was published openly. Should Chinese labs therefore not use transformers? Should US labs not try to prevent distillation of their models?
Ah brings back Halo 2 memories
Feels like Anthropic crying do as I say not as I do.
it's not complex. there's hundreds of billions of investor dollars counting on vendor lock in and walled gardens
If I'm a business and I need something done today, and bc Anthropic has the best model, there's a 99.9 chance it will be completed successfully for $1000. And using Deepseek there's a 70% chance it will, for $10 - you or me will go for the $10. Big businesses don't. Bc 1000 per task is nothing to them.
The real issue is that Deepseek has a 99.7% chance. So I can run it 10 times until it works and still pay 1/10 the money.
I don’t think it is intentional but this is actually quite bad for the western labs.
The entire booster narrative has been “look at how their revenue is growing! $10bn to $100bn ARR in under a year! This’ll be a multi-trillion IPO!” and the extrapolated future growth from $100bn to $500bn and $500bn to $1tn justified future investment… but that revenue was just because inference was expensive.
The revenue growth story is all that matters pre-IPO. If revenue falls from $100bn to $50bn that’s very very bad optics for OpenAI and Anthropic even if they are now profitable, it completely destroys the growth narrative.
"Thus the expert in battle moves the enemy, and is not moved by him."
They figured out a clever method for avoiding excessive training costs via distillation. That forces the hand of frontier labs to move faster, produce better models, etc. (to avoid embarrassment and 'falling behind'—all the while shouldering most of the cost), which they can just keep distilling—or applying other techniques against—much to the dismay of said frontier labs.
Checkmate.
I think this explains why they are open sourcing broadly. It's not to be nice. It's a strategic play by the Chinese government to help ensure there are many players in this race and not too much power accumulates to American labs (even if American labs benefit in the process)
I realize "going as fast as we can" is not the most popular position atm. But I'm far more interested in what good we can do than 10% apocalypse scenarios. I volunteer with a charity for childhood brain cancer and I do not want to see another 4 year old die. I'm willing to risk anything to stop this.
How co-designed are these optimizations with the model itself? I'd imagine you can't just stick post-training adapters onto existing architectures for these things, or am I wrong?
I really want to explore the inference space, but it seems like many of the inference optimizations are coming from model-hardware codesign. I don't seem to recall many generic "inference engine" optimizations since prefill/decode disagg a year ago.
This matters for me since I want to break in but the bar seems to be understanding the actual theory of the training process now too given the codesign happening, and I'm not the richest guy on the block lol
Because contrarily to the author's assumption, all labs, Western or not, have sufficient skills to discover the optimizations anyway, and publishing or not is not actually that important?
I'm glad people are saying this out loud, because that is what they want. Not for the good of the world, but for the good of their pockets.
I’m incredibly skeptical that OpenAI is spinning up custom ASICs for improved inference performance, but they never thought of optimizing KV cache until a tiny Chinese lab did it? Give me a break.
timeline suggests not.
Had western labs figured that out before, they would have used it to make kv caching cheaper before and not only now.
The burden of proof here is on western labs. But I doubt they'll try to lie that much.
I mean, there's a pretty big difference between labs publishing their research openly and a competitor utilizing it versus a lab breaking TOS to... hmmm, what's the word? steal data from a competitor?
>That’s because, unlike the Western companies, the Chinese are pretty much giving away their recipes.
Yeah, Western AI companies have never published their research. It's crazy how the Chinese had to independently develop the foundational technology that powers LLMs because Western companies simply never publish their research (I mean, as long as you ignore stuff like this <https://arxiv.org/abs/1706.03762>).
>The latest one shamelessly copied without acknowledgement is the breakthrough in KV cache optimizations that DeepSeek has generously shared with the world.
Thank you, generous corporation. I'm sorry that other corporations don't provide you free publicity for your selfless contributions to the world.
>Now I don’t know why they would freely give away such a breakthrough, but they just did
Well I'm glad the author finally got to their point. A very insightful analysis.
>They do seem to be a little embarrassed by the copying. Hence the silent releases without much pre-announcement for both Claude Opus 5.5 and GPT-6.1 Sol.
You have to be in pretty deep to infer this kind of emotion to these kinds of corporate activities.
>So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
Then why write this article? Why point out these things just to have no conclusion?
This article sucks. Even if you hate US AI labs and are all aboard Chinese labs producing open models, there's nothing of substance here. This is the loose draft that you hand to your LLM to finish for you, but it seems the author just forgot to do so.
Even if you're willing to characterize US AI labs as evil and selfish and Chinese AI labs as righteous and generous (which is already completely trivializing these dynamics to the extent that anybody over the age of 14 can likely identify is lacking nuance), you can at least put some effort into producing some hypotheses about why these dynamics are occurring. Of course, odds are if the author did try to articulate some hypothesis, they'd likely quickly realize that the narrative they're painting just doesn't hold up.
This is an extremely thin analysis that has obviously been voted to the top of the homepage because HN hates the big labs.
Is the thinking that the day or so between US systems achieving ASI and Chinese systems doing the same, we'll figure out a way to neutralize them indefinitely? Because otherwise, none of this makes much sense. And it only starts to swerve back to sanity if the assumption is that this isn't a race or competition, but instead a joint effort to achieve something good for humanity. But you can't really delta profit off that, can you?
To borrow your steam engine analogy, if local LLMs get as good as a Toyota Prius, even if OpenAI / Anthropic offer Ferraris, most people will be happy with their Prius as their daily driver.
Similarly, if the big labs start raising prices or cutting usage, you won't be able to use it as much as you want -- whereas a local LLM will run all day every day without costing you any extra money.
So right now you are right, but who knows how long that will last.
People did use global agreements and regulation to fix the ozone hole, though, so I think there’s a chance.
I predict that the AI scaremongering will fizzle out when the bubble bursts. There will still be die-hard believers but the public will lose interest.
I don’t see how a stock market crash will make AI-related concerns go away. There was a dot-com crash but the Internet just kept getting bigger and causing more problems. In many ways security has improved, but we worry more than ever about social media, etc.
You can do this with cars, tools, computers, ... whatever you want. So, no, I think your point is wrong.
Now, what I want to regulate are accordions.
Haven’t tried getting a gun in California. How bad is it? How could it be improved?
Other countries have governments that have earned that level of trust. I believe the US could get there eventually, but it will take a very long time because it has a very long way to go.
Replace “gun” with anything and you will see how your comment falls apart.
What’s next? A registry for food purchases? Your beer gut is starting to show.
We do have lots of food safety regulation, which has more to do with selling food.
Worse for whom?
The only effective defense against predatory corporate and government AI is personal protective AI.
Anything else is unilateral disarmament. It's the only way individuals can survive in the worse case scenario.
> gun nuts
Guns are different. They can't protect you against the government, contrary to gun nut claims.
You ask about distillation but I wonder, is there any training datasets (~ TB-order) available that startup folks in SV use or is it so that everyone has to create their own scraping pipeline ?
Or did they not pull back when their models allegedly became highly capable, with the whole mythos debacle ?
https://www.anthropic.com/research/glm-5-3-and-the-spread-of...
It's VERY clear that the US companies are trying to push for regulation to kill open models and open weights. I see this as much more hostile and authoritarian response than what we're seeing come out of China right now.
So is China going to always publish in the open? No clue. But right now they're modeling much better behavior.
The point is get what you can from both to develop open models, data and tools.
Local inference will have a boom of cheap, powerful, and available cards at some point (even if it isn’t until 2028/2029). At some point the hyperscalers, and frontier labs, will face the capex problems that everyone talks about, and NVidia, AMD, Apple, and Intel will want to keep selling products.
Powerful, by today’s standard, local inference needs to be accessible to really unlock the “AI” economy long term. It’s just like how the move from mainframes to the PC 40ish years ago unlocked the “computer revolution.”
Big businesses might pay $1000 vs $60 for certain tasks, but that won't work out well at scale.
Most folks I know can choose from any of the big labs or open weight models and they get billed internally for tokens against their budget. There’s little incentive to no switch to the lower cost closers.
This setup is a nightmare scenario for the big labs trying to execute the traditional enterprise sales plays. Those only work if your product is sticky and AI models are one of the least sticky things in the history of tech.
Also, is it really 99.9% vs 70%, or 99.9% vs 99%?
An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.
Anthropic’s anger here seems mostly rooted in their annoyance that this exposes they don’t really have core IP that’s not just easily replicated. And that’s clearly a problem for a deeply unprofitable company trying to convince people they’re worth $2 trillion.
The original authors of all the text, creators of the media and developers of the software did far more work than Anthropic.
This isn’t meant as a moral argument, just musing about the relative cost comparison.
With my apologies to Brewster Kahle, "Universal Access to All Knowledge."
I think a good litmus test here would be if Anthropic were to not care about distilling their models when the distillers keep the resulting models closed-source and sell tokens via an API. If they cared only about security concerns and not about people profiting off of their work, then they should be publicly fine with this and only protest against it going into open-weights models.
I see absolutely no distinction between the two, aside from minor technical approaches to gathering the content.
And writing a book requires many more resources than what anthropic does
The word itself is the pivot, not anything else.
I don't know anyone with even a passing understanding of how LLM training works that thinks that is the appropriate analogy.
Can you elaborate on that? I mean my direct answer would be no, of course not. But what is your reasoning?
I never take them seriously, I just assume they are coming from countries that don't understand how capitalism works or are operating out of bad faith. The underlying reality of the market is always changing and needs are always changing. Some AI companies will fail, that is a given. Remember alta-vista? Yahoo? Did search go away? How about Microsoft phones? Nokia? Motorola?
OpenAI and Anthropic are not in the inference business. That is a commodity. They need to sell products and solutions.
Inference will be too cheap long term to make money because it is being commoditized and customers will start to care about results and not just be wowed by impressive technology.
And of course this technology will continue to exist but that is irrelevant to the business. OpenAI investors don’t care if LLMs exist in 10 years, they care if their investment in OpenAI has made money.
A training set of 15 trillion tokens is 10 trillion words.
A penny a word is cheaper than the cheapest beginner freelance writer.
That makes a training set of 10 trillion words cost $100B.
Lots of assumptions there for sure, but we're certainly in the ballpark you are describing.
The totality of the content on internet is worth several orders of magnitude more.
Is that not what the foundation models are? A new reorganization of existing knowledge?
amelius•48m ago
Any ideas?
curuinor•44m ago
Because of the basic huge recession going on in China, you can't actually make money in China doing China things. So they gotta gird up their export stuff and try to export. That entails strong relations with American companies, American PR, English stuff, etc.
If you want an essay about this from a VC, read this one
https://earnedintuition.substack.com/p/involution-without-ex...
dabedee•38m ago
curuinor•33m ago
ajkjk•31m ago
iamnothere•29m ago
Enshittification and related problems can be a result of market forces just as much as they can be a result of monopoly/duopoly or a small cartel. Excess competition sometimes results in all firms scraping the barrel to squeeze out pennies, especially with technology (such as large online marketplaces) making pricing more transparent.
Marx actually predicted that ever-intensifying competition would destroy markets through overproduction, although he did not use the term involution.
tancop•11m ago
The reason it doesn't work like that IRL is centralized marketplaces. If the winning strategy on Alibaba is low prices, bad quality and botted reviews to compensate then every seller has to do it to survive, because they can't get buyers outside the platform. That's not excess competition. It's a lack of competition just on a different level.
thrawa8387336•37m ago
curuinor•34m ago
luke5441•29m ago
Not that OpenAI, Anthropic or SpaceX aren't doing the same.
googaar•27m ago
bilbo0s•22m ago
Do you do business in China?
I'm curious what you mean by this? Because in my experience, you can only do business in China by doing "China" things.
I'd be interested in picking your brain as to how you get around those issues?
curuinor•18m ago
I'm talking like, getting 100x, VC sized returns. Of course you can sell widgets in China, it's a major world economy.
foul•43m ago
pj_mukh•41m ago
twoodfin•33m ago
Once upon a time, everyone had a secret sauce in network or data encoding or query optimization, but in the last ~10 years computational physics and economics have basically decided the “correct” architecture and everyone (including OSS) has converged.
carbonguy•38m ago
mpalmer•36m ago
TrackerFF•35m ago
Basically, western labs are in it for the money / commercial monopoly. Chinese labs are in it for the tech? As long as they can keep distilling models, and get access to research other ways, they benefit. And if they can push western labs forward, they'll benefit from that themselves.
jollyllama•35m ago
corford•35m ago
chrismarlow9•30m ago
I can't even fathom the trend these days of "we don't review the code" from security team perspective.
Just my guess though.
Windchaser•29m ago
Unpopular, maybe, but what about the normal reasons? The researchers are looking to make a name for themselves, and/or they genuinely care about AI advancement.
feverzsj•28m ago
The weird ideology here is to dominate the market at ANY COST, even it benefits the opponents.
teekert•28m ago
Why did we (the west) ever start open sourcing anything? Maybe we just like sharing? Maybe humanity only grows on pre-competitive layers like Linux and clean water. Maybe, the chinese government is closer to their people, and does not let large companies influence them and just doesn't like closed private hyperscalers with a lot of power?
(Some points assume the government has a role in the openness, which I think is likely)
Catloafdev•27m ago
thefourthchime•26m ago
We don't know either way, so I find the whole thing silly to speculate on.
seydor•25m ago
audunw•22m ago
Put another way: if they were not cheaper and open, they would simply not be competitive. They would already be dead.
I don’t think this ends well for the Chinese labs. This is going pretty much like I thought. Western labs is just copying their improvements (I don’t think publishing the techniques matter here.. they’d just hire to gain the knowledge or figure it out themselves), and they have access to more GPUs and have better branding, so in the end where can the Chinese labs compete? Even lower cost? Open weights? I’m not sure open is a sustainable way to compete either. Eventually there will be some fully open source AI models that cuts out that avenue of competition as well.
HeavenFox•18m ago
micromacrofoot•10m ago
They're building bridges over the moats that companies with far too much US investment are trying to build, and if they do it continually it can help destabilize the US economy.
nater5000•8m ago
ozgung•1m ago