Curious how the memory bandwidth actually holds up under sustained inference loads compared to the spec sheet numbers.
varispeed•25m ago
Aren't these specs already outdated? Seems meh. I would think players would like to bring something new to the table instead of bringing what was okay two years ago or so.
Give me 1TB RAM and 5TB/s or something that exciting.
tempest_•14m ago
5 year old GPUs are still at crazy prices.
Plenty of businesses want in local inference and are priced out.
entrope•9m ago
For most purposes, this seems reasonable? This looks like a single-height card, so fitting four of these in a workstation or server would give 512 GB RAM and 2 TB/s total bandwidth. That's pretty attractive for a lot of current LLMs, even if one wishes for 1 TB capacity at 5 TB/s bandwidth.
aetherspawn•15m ago
Ok so it’s significantly worse than Apple M6 and we don’t know if it’s cheaper or not
dist-epoch•5m ago
Apple M6 has 80-100 TOPS.
Qualcomm Cloud AI 100 Ultra has 800 TOPS, almost 10 times more.
zitterbewegung•10m ago
Sort of interesting design It has half a terabyte of SRAM on the accelerator with a 128 GB of RAM replacing HBM with SRAM but keeping regular RAM.
2001zhaozhao•9m ago
half a gigabyte not terabyte lol
tokepi•7m ago
If the specs seem outdated, it’s because this chip launched in 2024. It’s available at cloud providers including Cirrascale
pearl_panel5•27m ago