Show HN: GoModel – an open-source AI gateway in Go; 44x lighter than LiteLLM

31•santiago-pl•1h ago

Hi, I’m Jakub, a solo founder based in Warsaw.

I’ve been building GoModel since December with a couple of contributors. It's an open-source AI gateway that sits between your app and model providers like OpenAI, Anthropic or others.

I built it for my startup to solve a few problems:

  - track AI usage and cost per client or team
  - switch models without changing app code
  - debug request flows more easily
  - reduce AI spendings with exact and semantic caching

How is it different?

  - ~17MB docker image
    - LiteLLM's image is more than 44x bigger ("docker.litellm.ai/berriai/litellm:latest" ~ 746 MB on amd64)
  - request workflow is visible and easy to inspect    
  - config is environment-variable-first by default

I'm posting now partly because of the recent LiteLLM supply-chain attack. Their team handled it impressively well, but some people are looking at alternatives anyway, and GoModel is one.

Website: https://gomodel.enterpilot.io

Any feedback is appreciated.

Comments

anilgulecha•48m ago

how does this compare to bifrost - another golang router?

santiago-pl•21m ago

First of all, GoModel doesn't have a separate private repository behind a paywall/license.

It's more lightweight and simpler. The Bifrost docker image looks 4x larger, at least for now.

IMO GoModel is more convenient for debugging and for seeing how your request flows through different layers of AI Gateways in the Audit Logs.

anilgulecha•16m ago

That would be valuable if there's a commitment to never have a non-opensource offering under GoModel? If so, you can document it in the repo.

santiago-pl•12m ago

I would love to keep it open source forever, but I can't promise that for now. I've written a whole doc page about it if you're curious: https://gomodel.enterpilot.io/docs/about/license

Talderigi•34m ago

Curious how the semantic caching layer works.. are you embedding requests on the gateway side and doing a vector similarity lookup before proxying? And if so, how do you handle cache invalidation when the underlying model changes or gets updated?

giorgi_pro•6m ago

Hey, contributor here. That's right, GoModel embeds requests and does vector similarity lookup before proxying. Regarding the cache invalidation, there is no "purging" involved – the model is part of the namespace (params_hash includes the LLM model, path, guardrails hash, etc). TTL takes care of the cleanup later.

pjmlp•34m ago

Expectable, given that LiteLLM seems to be implemented in Python.

However kudos for the project, we need more alternatives in compiled languages.

santiago-pl•15m ago

Agree and thank you! Please let us know if you'd like to give it a try and if you miss any feature in GoModel.

Taint: Testing Artifact Integrity by Non-Repudiable Tampering

The Oil Market Breaking Point Is Here

Show HN: Too many browser tabs! I solved it with a simple electron app

Show HN: LennyFighter: A multiplayer fighting game on Cloudflare Durable Objects

The Role of a New Machine

Map 236 years of US constitutional power on a grid: where design meets reality

Alarms Sound over 'Technofascist' Palantir Manifesto

An AI bot is running a retail store. Is this the future?

CATL's new LFP battery can charge from 10 to 98% in less than 7 minutes

Rapidhash Unity Port

Vector math library codegen in Debug (2024)

Blue Origin's New Glenn Fails Third Launch, Jeopardizing NASA Moon Plans

EU Commission awards €180M tender for sovereign cloud to four European providers

Concluding the Arc Experiment (Email)

Show HN: Runner – A Better Claude Cowork

Former ransomware negotiator pleads guilty to BlackCat attacks

SuperDisk: The Better Floppy That Never Caught On

Show HN: My Silly Sun Server

Falling fertility, debt and AI: is the US headed toward a population crisis?

A Harness for Claude Code

Months of Flaky CI. The Root Cause Analysis Was in CloudTrail All Along

Show HN: Easily share games made by you/ your kids/ your dog etc.

Opus 4.7 isn't dumb, it's just lazy

Moving Past Bots vs. Humans

I made a research test suite for analysis of software metrics overtime

Why Are Palantir and OpenAI Scared of Alex Bores?

Designing web interfaces with Claude Code

Better (Free) GitHub Readme Widgets

The Luddites Were the First to Attack AI

Is your site agent-ready?