TScale – distributed training on consumer GPUs

92•zX41ZdbW•6h ago

Comments

ArtTimeInvestor•5h ago

Even with consumer GPUs, the AI stack is completely dependent on ASML, isn't it?

Thought experiment: What would happen if the Dutch government decided that AI is bad for mankind and shuts down ASML? Would the world be stuck in terms of AI? For how long?

TechDebtDevin•5h ago

ASML publishes most of the research and theres not much stopping people from building their own EUV lithography machines. Its just very very very hard and basically the equivalent of doing magic. China is making incredible progress on this front.

airstrike•4h ago

The problem with these things is that there are always trade secrets that aren't published anywhere. So you'd need to actually hire people with specific knowledge to be able to replicate it.

The world (and the West specifically) definitely needs to build redundancy ASAP here.

TechDebtDevin•17m ago

The new machines are 2-3 stories tall, require an Airbus to transport, have complexity on par with the worlds largest particle accelerators, if not more complex. Because of this, the supply chains are highly intertwined no one country and can isolate that supply chain. The Dutch can't build it without our contributions, and neither could we without theirs. Lots of moving parts here literally and figuratively.

bgnn•5h ago

That's a silly the thought. ASML isn't controlled by the Dutch government.

Also, everything in computing is dependent on semiconductors. ASML is just one player. There are tens of thousands companies involved in the industry and some of them are single suppliers of critical materials, machines or software. It's wrong to single out ASML.

mschuster91•3h ago

> ASML isn't controlled by the Dutch government.

Of course they are. The Dutch government is who ordered ASML to not export their brand new stuff to China.

wokkel•2h ago

Actually it was the usa presuring the Dutch government.

hotstickyballs•1h ago

Distinction without a difference

coredog64•41m ago

ASML licensed technologies from US companies during the development of EUV. That's what gives the US the leverage to do things like block sales to China.

TYMorningCoffee•5h ago

Can the inference piece be partitioned over multiple hosts?

Edit: algorithmed or partitioned in a way that overcomes the network bottleneck

happyPersonR•5h ago

Pretty sure llama.cpp can already do that

TYMorningCoffee•3h ago

I forgot to clarify dealing with the network bottleneck

Maxious•4h ago

> prima.cpp is a distributed implementation of llama.cpp that lets you run 70B-level LLMs on your everyday devices— laptops, desktops, phones, and tablets (GPU or no GPU, it’s all good). With it, you can run QwQ-32B, Qwen 2.5-72B, Llama 3-70B, or DeepSeek R1 70B right from your local home cluster!

https://github.com/Lizonghang/prima.cpp

zitterbewegung•5h ago

I'm trying to run this but fo.cpp doesn't exist in the repository. I made an issue see https://github.com/Foreseerr/TScale/issues/1

mdaniel•2h ago

I suspect this was prematurely published to HN and was in fact just someone's weekend project

https://github.com/Foreseerr/TScale/blob/aa2638c53c74dd33280...

and I struggle to think of what would lead one to the urge to implement a key=value config file parser in 2025 https://github.com/Foreseerr/TScale/blob/aa2638c53c74dd33280...

On top of that, people who do $(git add . && git commit -myolo) drive me crazy https://github.com/Foreseerr/TScale/blob/main/logs/125m_1T_f...

comex•1h ago

> and I struggle to think of what would lead one to the urge to implement a key=value config file parser in 2025

C/C++ culture never changes.

As many new build tools and package managers as people come up with, the ‘default’ environment is still one where adding dependencies is hard, so people roll their own utilities instead.

revskill•4h ago

Interesting that you put code in code folder, not src.

fizx•2h ago

What is this 1T index technique they seem so hyped about?

emorning3•2h ago

>> In this case we build a model with 1T index which we lookup for every token to make prediction with much smaller model. <<

This index seems to be used to minimize the size of models.

I'm familiar with term indexing as described in The Handbook of Automated Reasoning and I imagine that this index helps them recognize 'generalizations'.

In the way that a rewrite rule can be used to reduce an infinite number of expressions, not just a single expression, a generalization can be used to minimize models.

Generally, such an index would be some kind of prefix-tree.

Just a guess, guessing is fun

CORBA: Catching The Next Wave (1997)

NASA Proposal Would Shift Agency's Focus Away from Space Science

Digital Archaeology, Part II

Why are there no thunderstorms in the UK?

Re the Boundaries of Human Potential

Light-activated pacemaker is smaller than a grain of rice

The Robotics Threshold: China's Rise, America’s Reckoning

I'd rather read the prompt

It's time to give the AI doomsaying a rest

The complicated business of electing a Doge

Why is Windows consistently losing market share? (2024)

'Dangerous nonsense': AI-authored books about ADHD for sale on Amazon

Bitcoin's Op_return War II: Code Change Could Break Bitcoin Forever? [video]

Nouswise: Next-Gen of Search Interfaces

Slovenia's Unexpected Economic Success

Score high on renewable energy, while reducing CO₂ emissions by 2050

Show HN: Aye.so – Send digital letters with a 10-hour delivery delay

TypeScript Docs – The Modern TypeScript Documentation

The first code deployable biological computer

Show HN: 100.st – Dev utilities I built for format conversions and encoding

The Elements of Euclid (With Highlights)

BBC and Agatha Christie estate respond to 'deepfake' controversy

The landscape artist who makes her paint from pearls, crystals and volcanic dust

The Tiny Sidewalk Boxes That Help Make New York City's Tap Water So Good

I Owe It All to Community College

HTTP/3 Stream Dependency Cycle Exploit

Options for Building GraphRAG: Frameworks, Graph Databases, and Tools

LLMs as Unbiased Oracles

Show HN: Script to read and alaize Mario Kart Wii ghost replay files

Mahalanobis Distance