Nowadays, specially with MoE models you can run parts of the model on GPU and still get some speed up.
Do you have LLM paranoia? The moment you read something in this style suddenly your brain starts doubting whether you are consuming human generated content.
Like what does the second sentence even mean? Is it even a sentence? "The roofline math, the prompt-processing catch, the NPU red herring, and the owner-measured speeds."
LoganDark•1h ago
mountainriver•50m ago
This is likely the right path in the future but it isn’t there yet today
bahmboo•48m ago