Ask HN: What inference engine and KV offload engine do you use and why?
1•malikoyv•1h ago
I have the question to the guys in inference world, I am building some stuff connected to the inference in my work, and I started wondering, what do actually big guys use in their inference work?
Do you prefer standard (and what is standard lol) solutions like vLLM and LMCache, or you additionally add Mooncake, or use vllm native?
Happy to have a discussion what is better and why as well, thanks!