Ask HN: Is it time to run the LLM engines on the CPU?
3•roschdal•45m ago
Comments
dtagames•41m ago
You always could. It's just slower and less efficient because most of what a model query needs to do is matrix multiplication and the GPU is optimized for that.
bigyabai•29m ago
It depends, how much coal are you comfortable burning?
roschdal•14m ago
In the future we could have CPUs which are better suited for llms, with GPU-like properties built in.
dtagames•41m ago