Alibaba Cloud claims to reduce Nvidia GPU use by 82%

https://www.scmp.com/business/article/3329450/alibaba-cloud-claims-slash-nvidia-gpu-use-82-new-pooling-system

77•hd4•3h ago

Comments

hunglee2•2h ago

The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end have reason to thank the US for their civilisational gate keeping

rzerowan•2h ago

Fingers crossed for convergence rather than divergence in the technical standards.Although the way hings are going it looks like the 2 stacks will diverge sooner rather than later , with the US+ banning the use of CHN models while simultaneosly banning the export of it quasi-open models. We may very well end up in a situation like the old PAL vs NTSC video standard where the PAL(EU/Asia/AFrica) and NTSC(America's/Japan) gradually converged with the adoption of digital formats. Instead here would be a divergence based on geopolitical considerations.

hunglee2•2h ago

positive take: a bifurcated tech tree might give us (humanity) a better chance of faster advancement, as it would be a persistent A/B test in live environment. Where I would join you in the crossing of fingers is to ensure such A/B testing is competitive but not destructive. We may even evolve to a situation of complementarity, an American Ying vs the Chinese Yang. Lets hope so!

reliabilityguy•2h ago

Tbh this whole situation reminds of how Japan excelled in making a lot more with a lot less after WW2, e.g., fuel-efficient engines, light cars, etc. these constraints were not present in the US (and to some extent in Europe), and resulted in US cars being completely not competitive in non-US markets.

dataviz1000•2h ago

I've been in Chile, Peru, Colombia, Panama, and Costa Rica.

The streets are flooded with cheap Chinese cars and I see more BYD than American cars. If the car wasn't made in Japan or Korea which probably account for most of the cars, it was likely made in China. Moreover, I haven't been in countries with the closest ties to China.

sofixa•1h ago

> The streets are flooded with cheap Chinese cars and I see more BYD than American cars

This isn't surprising in any way, American "cars" (quotes because the vast majority of what American manufacturers pump out isn't cars, it's trucks) haven't been competitive in decades. The only globally competitive vehicles were developed in Europe by GM Europe (Opel, since sold to PSA now Stellantis) or Ford Europe (which axed all models bar the Puma). The rest is too big, expensive and inefficient from the vast majority of uses. Tariffs and good marketing keep American car manufacturers in business in the US, but those don't work in most other markets.

The more appropriate comparison is with European automakers such as VW Group, Stellantis (Peugeot, Citroën, DS, Fiat, Chrysler, Dodge, Ram), Renault. And there too BYD is winning as well in mosy countries, but at least there's a comparison possible.

tsunamifury•32m ago

The premature optimizer is never the innovator.

Japan eventually stopped that role and their products improved greatly.

segmondy•2h ago

may backfire? it's a bit too late for that.

go to 2024, western labs were crushing it.

it's now 2025, and from china, we have deepseek, qwen, kimi, glm, ernie and many more capable models keeping up with western labs. there are actually now more chinese labs releasing sota models than western labs.

hunglee2•2h ago

too early to call a winner, though it is disappointing to see US withdrawal from open source. Still the main outcome of open source is distribution / diffusion of the idea, so it will inevitably mean US open source will come back, hopefully via some grass roots maniac, there will be a Linus-like character emerge at some point

segmondy•2h ago

i'm not calling a winner, i'm just saying that the chinese have caught up despite the embargo. google, openai & anthrophic have phenomenal models. i stopped using openai & anthropic after they called for open weight/source regulation. i use google because they offer gemma and i got a year gemini-pro subscription for free, use openai gpt-oss-120b since i can run it at home, and the only model i currently pay for is a chinese model.

mixologist•1h ago

user growth has slowed. the technology that should help users is only being pushed from the top, while users refuse to use it. openai pivoted to porn.

does it really feel like they have a chance to recover all the expenses in the future?

crypto grifters pivoted to ai and, same as last time, normal people don’t want to have anything to do with them.

considering the amount of money burned on this garbage, i think we can at least declare a looser.

dlisboa•2h ago

History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years.

In many senses there's hubris in the western* view of China accomplishments: most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the names of AI researchers there's a strong pattern even if some are currently plying their trade in the west.

---

* I hate the term "western" because some "westeners" use it to separated what they think are "civilized" from "uncivilized", hence for them LATAM is not "western" even though everything about LATAM countries is western.

achierius•2h ago

> most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the names of AI researchers there's a strong pattern even if some are currently plying their trade in the west.

While I don't disagree with your overall point, it's important to recognize that this is only a phenomenon of the last ~30 years, and to avoid falling into the trapn of Han racial chauvinism. E.g. there were ~no Chinese scientists in Germany in the 70s but they were heavily innovating nevertheless.

dlisboa•1h ago

Absolutely. China obviously has a longer history with innovation but they like to make it seem everything was invented by them at some point in the past. I'd say newer technology is where China has had a bigger impact.

Consequently newer tech is precisely where global cooperation is most required so no country can really do it by themselves. We could even say no country, western or otherwise, has been doing it on their own for the past 500 years or so but alas...

tsunamifury•43m ago

It’s more helpful to think of China as an accelerant rather than an innovator in this position.

notepad0x90•2h ago

western is a cultural term derived from a geographic one. The US is also not 'western' strictly geographically as it is not in western europe, neither is australia. But they both originated from Britain's empire and share in it's cultural ancestry. It means "western europe and it's cultural derivatives". Spain and Portugal's empire fell away long before britain and france's and they don't have similar geopolitical relations like NATO, so it's hard to consider their former colonies/upstarts part of the same sphere of cultural influence.

China for sure will catch up, the question is what they will do with it. They're not ambitious like the US/West. The US wanted influence all over the world as an extension of the cold war and to keep economic interests safeguarded. But China just doesn't operate that way. They're more hands-off. They could be opening up alibaba cloud datacenters all over the US, offering it as an AWS/Azure alternative, funding tons of startups all over europe, the US,etc... to exert their influence, but they won't. They have a more long-term low-and-slow approach to global domination. The "100 year marathon" as they called it, which they'll win for sure.

China's greatest weakness is not just their lack of ambition,but their command-economy. They're doing capitalism but with central control of the economy. It intertwines government policy with corporate policy, making it harder to do business overseas (like with bytedance/tiktok).

tsunamifury•42m ago

False.

Westernism is broadly an extension of the academic notion of classicism, starting in Egypt and then Greece Rome and into Europe and the Americas.

zawaideh•1h ago

Re: Western. A similar thing plays out when the term "international community" is used in news. It refers to the US and its major allies which means US, Canada, Western Europe, Japan, Australia and New Zealand more or less.

newyankee•47m ago

Essentially countries that were developed prior to 1990 or so , although South Korea is a tricky case today going by this definition, as are Taiwan, Hongkong and Singapore

tsunamifury•45m ago

Yes community refers to whose who participate in community.

How is this hard to understand?

Broadly speaking coast de ivory and the like is not a participant in the international community.

onlyrealcuzzo•1h ago

> History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years.

It's worked for a very long time for aircraft.

China has been pushing to build its own aircraft for >23 years. It took 14 years for COMAC to get its first regional jet flying commercial flights on a Chinese airline, and 21 years to get a narrow-body plane flying a commercial flight on a Chinese airline.

If for no technical reasons and purely political, COMAC may still be decades away from being able to fly to most of the world.

Likewise, in ~5 years, China may be able to build Chips that are as good as Nvidia after Nvidia's 90% profit margin - i.e. they are 1/10th as good for the price - but since they can buy them for cost - they're they same price for performance and good enough.

If for purely political reasons, China may never be able to export these chips to most of the world - which limits their scale - which makes it harder to make them cost effective compared to Western chips.

sofixa•1h ago

> China has been pushing to build its own aircraft for >23 years. It took 14 years for COMAC to get its first regional jet flying commercial flights on a Chinese airline, and 21 years to get a narrow-body plane flying a commercial flight on a Chinese airline

And both those planes have a strong dependency on "western" components that won't be overcome before the 2030s, and even then, they're around a generation behind.

Yoric•1h ago

> If for purely political reasons, China may never be able to export these chips to most of the world - which limits their scale - which makes it harder to make them cost effective compared to Western chips.

Note that this happens at the same time the US is breaking up its own alliances, so as of this writing, there's no such thing as certainty about politics.

rayiner•1h ago

Nobody thinks the Japanese aren’t “civilized.” “Western” is just a euphemism for “rich and orderly.”

bad_haircut72•1h ago

Its more about democracy and adhering to the global (set up by America post WW2) system of laws and trade.

rayiner•1h ago

I think most people considered Spain a "western" country even in 1970 when it was controlled by Franco.

chuckadams•52m ago

I find "western" is often used disparagingly, as in those mysterious orientals have some deep wisdom that transcends normal logic and reason that our "western thought" can't grasp. Declaring such a split is the underpinning of a whole lot of woo-woo beliefs.

tsunamifury•51m ago

It’s helpful to think of westernism as a platonic ideal. Individually derived reason and virtue, superior to state and sometimes ‘gods’ as a tradition to drive up the total survivability, richness, and stability of the community.

Concepts that enable the individual should empower a chosen configuration of society not the other way around.

Contrast this with non westernism where either education of the individual is not valued or the state is the primary goal over the individual.

I’ve worked with states governments and individuals around the world for 20 years and find this very useful definition. What’s confusing is the nations who have half adopted westernism but don’t fully due to either caste systems or government dominated thinking.

It’s an arrow towards rationalism over tradition, individualism over collectivism, flatness over hierarchy, and future over past. But only the limit of the resources any given society has.

caycep•41m ago

this is true for anyone - create challenges, and you optimize efficiency elsewhere.

Also, isn't this the usual path to better computer science? Reducing computation needs by making better/more efficient algorithms? The whole "trillions of dollars of brute force GPU strength" proposed by Altman, Nadella, Musk et al just seems to reinforce that these are business people at heart, not engineers/computer scientists...

notepad0x90•2h ago

I think anti-immigrant rhetoric will have the most impact against the US. A lot of the people innovating on this stuff are being maligned and leaving in droves.

Aside from geography, attracting talent from all over the world is the one edge the US has a nation over countries like China. But now the US is trying to be xenophobic like China, restrict tech import/export like China but compete against 10x population and lack of similar levels of internal strife and fissures.

The world, even Europe is looking for a new country to take on a leader/superpower role. China isn't there yet, but it might get there in a few years after their next-gen fighter jets and catching up to ASML.

But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india.

dlisboa•1h ago

> But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india.

That's a strength. Them not having interest in global domination and regime change other than their backyard is what allows them to easily make partners in Africa and LATAM, the most important regions for raw materials.

notepad0x90•1h ago

You would think so, but historically that's why they never became more than a regional power. Empires for millennia craved trade with China but only the mongols from that region made it all the way to western europe in their invasions.

It is a strength, if their goal is to have a stable and prosperous country long term, and that seems to be what they want. good for them. But nature abhors a vacuum, so there will always be an empire at the top of the food chain. Such empires want to maximize wealth for their people and secure them against threats, that's why invasions and exploitation of weaker countries happens. That game hasn't changed. Friendly relations work, until you need a lot of resources from a country that doesn't want to give it up. Or, like with the US, when they're opening up military bases next to your borders and you need a buffer state. Or, when naval blockades and sanctions are being enforced against your country for not complying with extra-sovereign demands.

History shows that countries content with what they have collapse or weaken very quickly.

China will have a population crisis in a few decades for example, and it won't have the large manufacturing base and its people will be too used to luxuries to go back to slaving for western countries for pennies. Keep in mind that the current china itself is so great and prosperous because of all the invasions it did against western china and satellite states like Vietnam and north Korea (the US isn't special in this regard).

lossolo•46m ago

> But nature abhors a vacuum, so there will always be an empire at the top of the food chain

The world has been bipolar and multipolar before in history, and it can be again. The unipolar period of American dominance is ending.

OrvalWintermute•1h ago

if you've been tracking the shark deals they give countries for loans, I think you'd recant what you just said.

"while the CCP accuses the West of predatory interest rates, the average Chinese rescue loan carries an interest rate of about 5 percent, more than double the IMF’s standard 2 percent. As of Oct. 1, 2025, despite higher U.S. interest rates, the IMF’s Special Drawing Rights lending rate stands at only 3.41 percent, still significantly lower than what China charges struggling nations for so-called relief."

These countries paying these loans are the ones least able to pay them back, and at more than double IMF loans, they are really putting them in a vise.

ikidd•5m ago

>Them not having interest in global domination and regime change

I don't even know where to begin with that one.

bkandel•1h ago

China's greatest weakness is that their working-age population has already peaked and is in the process of plummeting, which will continue over the coming decades.

notepad0x90•1h ago

Yes, and being content and lacking ambition isn't good. Expansionism and immigration can solve that, but they're culturally stagnant in that regard.

Without immigration, the US would have faced the same problems.

csomar•1h ago

> But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india.

How can they have international hegemony before they clear their regional order? China is more interested in aligning Taiwan than invading; though it’ll probably invade if it can’t align it diplomatically.

China is probably not interested in continuing the current Western-style order but to implement their own sino-stuff. At least with the CCP at the helm.

rayiner•17m ago

> But now the US is trying to … compete against 10x population and lack of similar levels of internal strife and fissures.

I can’t tell whether you think the anti-immigration stance is a good thing or bad thing.

IT4MD•1h ago

I believe this is an Pollyanna take on AI. There is nothing about humans that tells us humans will bring AI to fruition for the other humans and a mountain of evidence showing how it will be used to abuse humans instead....for profits/power/whatever horse shit the masters of the universe have decided upon.

amelius•1h ago

Another outcome may be that we now have to learn Chinese to understand their datasheets ...

myth_drannon•59m ago

China's innovation relies on the stolen western IP, without it, China is nothing. Also USSR/Russia is no longer a scientific powerhouse that can supply China with some military innovation. A dictatorship combined with cheap labour it 100% guarantees that the country's innovation is stunted, no matter what the Chinese propaganda claims.

Hikikomori•47m ago

And the US has never stolen IP?

myth_drannon•41m ago

Corporate espionage is ever present but it is criminalized. The only time US as a country did that you can say "stole IP" was after WII when it took Nazi rocket scientists and technology. China is the opposite; stealing tech is done by the state apparatus (same was done by USSR and reverse engineering computers for example).

Frankly I'm not surprised that this is done, probably if US was so behind it would have done the same to reduce the gap. Everyone is trying to survive and outsmart and outwit the other, instead of collaborating.

currydove•30m ago

Negative.

https://interestingengineering.com/innovation/the-spies-who-...

https://yawboadu.substack.com/p/stealing-success-how-ip-thef...

https://ipwatchdog.com/2017/07/05/americas-industrial-revolu...

g8oz•23m ago

You are mistaken about American intellectual property theft. They engaged in extensive IP theft from Britain in the 18th and 19th century with the encouragement of the government. See https://interestingengineering.com/innovation/the-spies-who-...

Also during World War I the American government seized German chemical patents thereby launching the American chemical industry. So that is an example of theft by the state apparatus.

lossolo•13m ago

Every single sentence you wrote is untrue and can be disproven by empirical evidence. You can learn about it here[1][2].

1. https://itif.org/publications/2024/09/16/china-is-rapidly-be...

2. https://www.economist.com/science-and-technology/2024/06/12/...

belter•40m ago

China is a nation of engineers...The US has been relying in on H-1B immigrants. Science is under attack. The truth is the US already lost: https://youtu.be/whVlI6H4d-4

majke•2h ago

better link https://www.tomshardware.com/tech-industry/semiconductors/al...

paper https://dl.acm.org/doi/10.1145/3731569.3764815

braza•2h ago

Does someone know if there's some equivalent of those engineering/research blogs for Chinese companies?

I used to follow the ones from Western companies, but honestly, after some point in time, I would like to see some cases from what I consider is a good benchmark for everyone that does not work in FAANG in terms of engineering.

djoldman•2h ago

Key paragraph:

> However, a small handful of models such as Alibaba’s Qwen and DeepSeek are most popular for inference, with most other models only sporadically called upon. This leads to resource inefficiency, with 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found.

throwaway48476•1h ago

Its easy enough for a a well resourced entity to take a pre trained model and deploy it on new hardware to save on the NVDA tax. It's far less likely for research and model training to happen outside the mature NVDA ecosystem.

muddi900•1h ago

I claim to be the most handsomest man alive.

Not everything has to be taken at face value. Especially when published by state media. It seems not growing up in a controlled media environment has made Americans ignorant of a fundamental concept: lying.

throawayonthe•55m ago

scmp is kinda the opposite of state media lol

larus_canus•45m ago

Ah, yes, the American media environment, which is internationally famous for not lying.

dotnet00•40m ago

This is such a popular coping tactic from Americans when it comes to facing actual competition, especially from China. Everything they do must either be a lie or just stolen American technology, as if there's something inherently special about Americans that no one else has.

lossolo•5m ago

It comes from what people are taught in schools and from their own self perception. When those beliefs about American exceptionalism are challenged, cognitive dissonance kicks in.

https://en.wikipedia.org/wiki/American_exceptionalism

kilotaras•51m ago

Alibaba Cloud claims to reduce Nvidia GPU used for serving unpopular models by 82% (emphasis mine)

> 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found

Instead of 1192 GPUs they now use 213 for serving those requests.

yorwba•29m ago

Not really, Figure 1(a) of the paper says that the 17.7% are relative to a total of 30k GPUs (i.e. 5310 GPUs for handling those 1.35% of requests) and the reduction is measured in a smaller beta deployment with only 47 different models (vs. the 733 "cold" models overall.) Naïve extrapolation by model count suggests they would need 3321 GPUs to serve all cold models, a 37.5% reduction to before. (Or 6.6% reduction of the full 30k-GPU cluster.)

BERT Is Just a Single Text Diffusion Step

Commodore 64 Ultimate

DeepSeek OCR

Space Elevator

Servo v0.0.1 Released

Matrix Conference 2025 Highlights

How to stop Linux threads cleanly

Docker Systems Status: Full Service Disruption

Anthropic and Cursor Spend This Much on Amazon Web Services

Modeling Others' Minds as Code

Entire Linux Network stack diagram (2024)

Show HN: Playwright Skill for Claude Code – Less context than playwright-MCP

How to Enter a City Like a King

Pointer Pointer (2012)

AWS Multiple Services Down in us-east-1

The Peach meme: On CRTs, pixels and signal quality (again)

Forth: The programming language that writes itself

State-based vs Signal-based rendering

Qt Group Buys IAR Systems Group

AWS Outage: A Single Cloud Region Shouldn't Take Down the World. But It Did

Optimizing writes to OLAP using buffers (ClickHouse, Redpanda, MooseStack)

Fractal Imaginary Cubes

Novo Nordisk's Canadian Mistake

Major AWS Outage Happening

Introduction to reverse-engineering vintage synth firmware

Duke Nukem: Zero Hour N64 ROM Reverse-Engineering Project Hits 100%

Give Your Metrics an Expiry Date

Gleam OTP – Fault Tolerant Multicore Programs with Actors

Airliner hit by possible space debris

Major AWS outage takes down Fortnite, Alexa, Snapchat, and more

BERT Is Just a Single Text Diffusion Step

Commodore 64 Ultimate

DeepSeek OCR

Space Elevator

Servo v0.0.1 Released

Matrix Conference 2025 Highlights

How to stop Linux threads cleanly

Docker Systems Status: Full Service Disruption

Anthropic and Cursor Spend This Much on Amazon Web Services

Modeling Others' Minds as Code

Entire Linux Network stack diagram (2024)

Show HN: Playwright Skill for Claude Code – Less context than playwright-MCP

How to Enter a City Like a King

Pointer Pointer (2012)

AWS Multiple Services Down in us-east-1

The Peach meme: On CRTs, pixels and signal quality (again)

Forth: The programming language that writes itself

State-based vs Signal-based rendering

Qt Group Buys IAR Systems Group

AWS Outage: A Single Cloud Region Shouldn't Take Down the World. But It Did

Optimizing writes to OLAP using buffers (ClickHouse, Redpanda, MooseStack)

Fractal Imaginary Cubes

Novo Nordisk's Canadian Mistake

Major AWS Outage Happening

Introduction to reverse-engineering vintage synth firmware

Duke Nukem: Zero Hour N64 ROM Reverse-Engineering Project Hits 100%

Give Your Metrics an Expiry Date

Gleam OTP – Fault Tolerant Multicore Programs with Actors

Airliner hit by possible space debris

Major AWS outage takes down Fortnite, Alexa, Snapchat, and more

Alibaba Cloud claims to reduce Nvidia GPU use by 82%

Comments