frontpage.
newsnewestaskshowjobs

Open Source @Github

fp.

Open in hackernews

Show HN: Proxima serves 4x more requests with no hardware change on vLLM

https://github.com/Tenosra/Proxima
2•pythongiant•55m ago
hey everyone, i decided to make a vLLM plugin that implements the Star-KV paper. the results are quiet promising with a decode kernel thats faster than FA2 in higher batch sizes. would love any thoughts and recommendations

Comments

pythongiant•52m ago
Tenosra is all about making AI more efficient and resourceful you can read more abour it at: https://www.tenosra.com/proxima