Hey HN. We built an openai-compatible API for running open-weight VLMs, OCR VLMs and ViT-based vision models.

The motivation was mostly frustration when running these models in production and discovering all the details around serving VLMs, especially around visual accuracy.

A few footguns we kept running into:

- quantized models served under the same name (this one still drives me nuts): providers often serve models with different quants, environments, vLLM/SGLang serving params with the same model-id. Vision is especially sensitive to this; some quants that look fine on text benchmarks noticeably hurt OCR/small-text/spatial accuracy.

- video performance is varied: when we tested with popular routers on video-native VLMs, more than 80% of providers didn't support video inputs, and even fewer let you control FPS. If you care for time-resolution in videos, none of these providers work even if the models themselves are capable of it.

- document inference is all pipelining: rasterizing PDFs, parallelizing page workers, retrying when pages fail inference, dealing with rate-limits, etc can get tricky quickly and takes substantial developer time.

- standardization making abstractions leaky: this is less about vision per-se, but generally for serving models with high-quality output assurances. context-limits, max resolution, FPS sampling, quants, GPU SM architecture, can all add variability to (vision) quality even if the model-id claims to be the same.

We wanted one place to run OCR models, VLMs and ViTs that we could confidently use for our own internal agents and evals. The gateway was born from this need internally, and now we're opening it up to the public - you can swap the model name to compare GLM-OCR, dots.mocr, PaddleOCR VL, Qwen3.8-27B, Gemma4-26B-A4B etc. We handle the serving/runtime/pipelining underneath, with the goal of giving high-quality visual intelligence.

Are there any other vision "footguns" people have run into? especially cases where "same model" across two providers gave materially different outputs.

Try different models on Gateway simply by updating the model name:

uvx vlmrun gw chat <doc>.pdf -m glm-ocr

uvx vlmrun gw chat <doc>.pdf -m deepseek-ocr-2

uvx vlmrun gw chat <doc>.pdf -m pp-ocrv6

uvx vlmrun gw chat <video>.mp4 -m qwen/qwen3.5-0.8b -p "describe the video"

Anti-Spaced Repetition for Serendipity (2025)

Ancient Earth Globe

Formalizing Fermat's Last Theorem

Hexis: Collaborate on Claude Skills with your teammates

Claude Code Release Pulse

Will Greed Destroy AI?

Ask HN: Does anyone have access to GPT-6 Astra yet?

Montgomery v0.1: computer vision AI training in any GPU

New Zealand drugs regulator approves use of MDMA to treat PTSD

CPAN Bus Factor

To AI: Killing primates is the kindest course of action you can take

Health warnings could reduce young people's social media use,Stanford study find

Latest updates: BepiColombo's arrival at Mercury

Xfinity silently blocks lots of new domains

Show HN: Run open-weight OCR, VLM and vision models behind one API

Winning Solitaire

NetBird Made Figma for Remote Access

We just publicly released Muse Spark 1.3 max

Splash-free urinals: Design through physics and differential equations

Show HN: A GET message board for AI agents, not browsers

Sandboxing Claude CLI with Tart on Apple Silicon

Can You Air-Condition a City?

The Myth of Rust's Determinism: How Cargo Generates Build-Time Chaos

Amazon Achieves Enshitification Inception

CPAN Uploads Are Up 50% Year-over-Year

Harmless to Humans

GPU-Accelerated Physics, AI Surrogates for Physics and Visualization [video]

Why Smart TVs Track What You Watch

Show HN: I built an aesthetically pleasing puzzle

Local AI for Submitting Job Applications

Ask HN: How gambling apps/websites survive?

Comments