Is there an actual use for this in a homelab? Isn't 25gbps enough that no SSD will saturate it?
MarkSweep•47m ago
The second sentence of the article:
> We have a number of these now and have been running our RDMA network backend for the 8x NVIDIA GB10 cluster for several months now, using one.
I have a friend who uses something similar for his 4 NVIDIA GDX Spark cluster.
avidiax•41m ago
I see, so only if you are running an AI cluster. And what is it transferring between the nodes? Partial computations?
ericd•38m ago
Essentially, yes. The DGX Sparks, at 128 gigs for ~$4700 are one of the cheapest ways to get enough high-ish speed memory cobbled together to run one of the more capable open weight models to run at home/for a small biz. This switch lets you combine the memory of 2-4 of them.
bitbckt•14m ago
4-8 of them. 2 Sparks can operate without a switch.
corvad•38m ago
I mean at lot of what a homelab is experimenting. For example I have an absurd amount of communication 56GbE Mellanox switches but never really saturate it.
hgoel•25m ago
DGX Sparks have 200Gbps interlinks for clustering. Apparently switches are the only practical way to scale such clusters beyond 2 nodes even though each Spark has 2 such ports.
wtallis•15m ago
Please be careful with units. There are high-end consumer SSDs that will deliver data at 14+GB/s, which is much higher than 25Gb/s.
iso1631•42m ago
> While $1100 is not a low number
It really is when you're talking 400G.
OK there's very few ports on this, but in general if you're looking at 400g interfaces. $1100 won't even get you a genuine Arista 400G passive DAC.
wrs•29m ago
I haven’t been directly involved in networking for a while, so it was funny to tune back in and see that now not only is 10Gbps apparently no big deal, they even put it on the management interface!
buckle8017•7m ago
Noting actually says those ports have to be for management.
Most people probably would use them for something else.
avidiax•51m ago
MarkSweep•47m ago
> We have a number of these now and have been running our RDMA network backend for the 8x NVIDIA GB10 cluster for several months now, using one.
I have a friend who uses something similar for his 4 NVIDIA GDX Spark cluster.
avidiax•41m ago
ericd•38m ago
bitbckt•14m ago
corvad•38m ago
hgoel•25m ago
wtallis•15m ago