I made this small proxy when I built a small GPU cluster at home and wanted to share it with friends while keeping speed limits and token accounting.
I also put a lot of effort in making it work as quick as possible, and IMO the result is worth sharing!
It is not trying to be LiteLLM: in fact, I wanted to make it opposite, as small as possible, with a small amount of dependencies and without cloud connections by default.
Will be happy to hear your feedback :)
PS also installable with pip install smol-llm-proxy