Model list is a bit limited, and whole harness affects the speed, so the libs used in harness and their versions might be useful to show for each individual model setup.
pil0u•24m ago
I don't know what basis these numbers are estimated on, but my M2 Ultra 128GB runs almost twice as fast as what's reported for the M3 Ultra 512GB.
I wished there was such a tool before picking my hardware, it's useful. But the estimates needs to be somewhat correct.
albinstman•2m ago
Thank you for the feedback. What model/quant did you test so I can look in it?
hbogert•17m ago
Shame they are not including old crap like my RX580 which is doing some impressive stuff with Qwen 9B Models.
bozdemir•25m ago