In my recent post on GPU engineering for LLMs, I argue that obsessing over CUDA kernel engineering isn’t the best starting point—understanding the full system stack, from model definition to hardware limits, is far more critical.

System level thinking helps you spot whether you’re compute-bound, memory-bound, or communication-bound before diving into low-level optimizations.

But I’m curious: for engineers breaking into inference engineering, where do you recommend starting? Should newcomers focus on mastering profiling tools and frameworks like PyTorch or JAX first or jump headfirst into distributed systems right away?

Also, I downplay kernel engineering in the post but are any specific scenarios where hand-tuned kernels have been a game-changer for you?