It sounds like iroh enables distributed compute without having to finangle custom hardware.

I thought about this too, but the throughput over a network is incredibly slow. It’s not usable for interactive use.

cocompute.ai is already doing this really well.

I note the lack of performance information. I can only imagine it's much, much, slower than any other way to run a larger model (including, e.g. using system RAM and streaming some stuff from disk). Consumer networks, even 10gbit ethernet, are slow as hell compared to local RAM and even disks.

Are we talking 1 token per second for a split model? Less?

Does Mesh LLM encrypt the payload between nodes? Is it possible to read requests from other users?