Too bad you probably need something like an 8x B300 node to run this. Black Friday is coming soon though, but I fear this open intelligence will be pacing anyone's liquidity frontier so I'll probably keep trailing the frontier for a while with nvidia/qwen3.8-27B-NVFP4 on dual RTX PRO 4000 in TP=2. I can almost fit 450k context in there.
graykevinb•27m ago
AND developer cloud has an image for K3 running it on a single MI300x. Offloading to system ram can be quite decent.
I cant say how well that image works as I'm still waiting for them to approve my gpu access
vorticalbox•26m ago
Do you need 450k context?
I usually keep under 200k if I can help it for Claude/ open ai
And lower locally on my Mac to save on prefill.
LarsDu88•19m ago
Possible data center bans incoming + tons of GPUs sitting in warehouses due to lack of power infrastructure to power them.
Perhaps a flood of 8x B300 will be homelab ready in a few months
swingandamiss•11m ago
No, they'll be shipped off to Asia where technology is embraced.
zozbot234•16m ago
You could easily run it with SSD streaming on as little as 192GB RAM (The dense portion is around 115GB or so at native precision, and the remainder would fit KV and a cache for the most commonly referenced experts.) This fits Gorgon Halo, which is a reasonably priced platform. It would be slow, but still usable for high-value tasks.
__jf__•48m ago
> Advancing the Frontier of Open Intelligence.
Too bad you probably need something like an 8x B300 node to run this. Black Friday is coming soon though, but I fear this open intelligence will be pacing anyone's liquidity frontier so I'll probably keep trailing the frontier for a while with nvidia/qwen3.8-27B-NVFP4 on dual RTX PRO 4000 in TP=2. I can almost fit 450k context in there.
graykevinb•27m ago
I cant say how well that image works as I'm still waiting for them to approve my gpu access
vorticalbox•26m ago
I usually keep under 200k if I can help it for Claude/ open ai
And lower locally on my Mac to save on prefill.
LarsDu88•19m ago
Perhaps a flood of 8x B300 will be homelab ready in a few months
swingandamiss•11m ago
zozbot234•16m ago