Thoughts on any-llm-gateway versus litellm-proxy?

litellm is a great library, but one team using litellm-proxy reported having many issues with it to me. I haven't tried it yet.

There is also PydanticAI Gateway (https://ai.pydantic.dev/gateway/). I use it with the PydanticAI framework and it's quite nice.

Any-LLM-gateway helps efficiently manage LLM usage and costs, providing control over access while making AI integration safer and more scalable for teams.

Any-LLM-gateway makes it easier to control LLM access and spending, helping teams optimize costs while maintaining secure and efficient AI usage.

This service (llm proxy to all providers) are a dime-a-dozen

This one has very little on monitoring and no reference to OTEL in the docs

Interested to see how this stacks up against Bifrost (fast but many features paywalled) and LiteLLM Proxy (featureful but garbage code quality). Especially if it gets a web admin / reporting frontend and high availability.

I'm conflicted on what Mozilla is doing here. On the one hand, it is nice that they are getting involved but com'on, dont you all have Firefox to work on?

This is a classic case of an over enthusiastic engineer who says yes / raises hand to everything, but doesnt do any one thing properly. At some point, you have to sit down and tell them to focus on one thing and do it properly.