I've spent most of this year working on GPU inference, and I put together a short course to share what I've learned.
It starts with the basics and works up to how inference is actually served, things like continuous batching, goodput, and disaggregated prefill and decode. I'll soon add a lesson that shows how to run adaptive disaggregated inference on a consumer GPU!
The lessons are around 5-10min each, include interactive diagrams and are fully voice narrated.
No signup or fees.
I'd love to hear what you guys think, and I'm happy to answer any questions!