I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp).

It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency

Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test

The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including the growing number of third-party “proxy” or “cheap LLM API” services.

It supports: • OpenAI-compatible APIs (official + proxies) • Claude (via Anthropic) • Local endpoints (custom/self-hosted)

You can also self-host it with docker-compose. Config is clean, adding a new provider only requires a simple plugin-style addition.

Would love feedback, PRs, or even test reports from APIs you’re using. Especially interested in how some lesser-known services compare.