This submission is a tale about how I launched an unlimited LLM provider to about 60 hyped people on the waitlist, then immediately served them a fully dysfunctional death-loop model, and how most people, very reasonably, disappeared, but thanks to a few extremely nice people stuck around anyway, we kept the project alive and its still pretty chaotic but gaining traction.

To back up a little bit-- I believe that the whole point of AI agents is that they should keep working. They should read files, retry, search, code, summarize, run tools, and loop until the job is done. When your employer is paying for it, who cares about cost, but when it comes to my personal money/hobbies, if every loop feels like a tiny financial event, you start babysitting the agent instead of using it, and its not fun.

On the other hand, metered pricing makes me worry about using too much. Usage subscriptions make me feel like I need to use every last magical % or I'm are "wasting it". If only an unlimited provider existed....

Then I joined the AMD developer program - I got some credits to spin up my own MI300x and started tinkering with vllm/sglang inference serving on AMD.

After learning about AMD MI300x , i did some napkin math:

Renting MI300x at 2.00 an hour = ~$1500 a month . It can probably support about 150 users using a small MOE model, like qwen-35b-3a , maybe more.

1500 / 150= $10.00 per month, and we all get to play with agents for a small price.

You can oversubscribe a bit, so i landed on $6 per month, per user, for 2x generation slots, 128k context, no token limits, no rate limits.

I built the site, router, made a waitlist, and then over-optimized the MI300x to the point where vllm bench had like 3k+ output and 40k+ throughput.... But i didn't test the final config/serve commands... And that's where i did a disaster launch. You couldn't prompt the thing without it looping or bugging out, it was cursed. And that's where we lost alot of people.

Luckily, my buddy had a few 3090s, so he threw me a life boat and began hosting qwen for us on 2x 3090s and we finally had an operational model that wasn't costing $2.00 an hour for our whopping 3 users.

We started gaining a more users, so we moved up to 4x 3090s. Which we have plenty of room for more users, but even so, since then:

we've configured vllm wrong like 15 times a GPU died we lost power I made a bunch of one-click starts for openclaw,hermes,pi-mono and none of them really work right and that probably drives people away. Those are still on our site right now.

...but people that know what they are doing seem to really be liking the price point. All in all we have like 98% up time. Its been about a month. We've both learned a ton, even already having backgrounds in SWE/SE/AI , being on the hook for a couple paying users forced us to really focus on delivering them a good product. And now i think we might be close to paying the power/hosting bill so we're not operating at a loss (if u include 3090 capex were still at aloss).

Our break-even point is moving to the cloud to max out a MI300x, which is now tuned and ready to go once we get the users.

And im finding in some areas, subscribing to our service is cheaper than running the model (but as someone who loves local models, i totally get it).

Since then, I've been working on a desktop agent that actually works with small models like qwen -- thats going to replace the broken 1 click starts. It's barebones, but its something out of the box that just works. I made it open source, you can see what im talking about here: https://github.com/yolo-auto-org/yolo-auto-desktop , we're at yolo-auto.com and we have an abysmal free tier to prove it works!

Anyway, hope you got a laugh or found it interesting! Drop a question if you have any.

New Brunswick woman sues OpenAI, alleging ChatGPT led to daughter's death

Don't trust large context windows

China Is Propping Up the World Economy by Importing Less Oil

Introduction to (Multimodal) LLM-as-a-Judge

Mining And refining: uranium and plutonium (2024)

How an astrophysicist uses Codex to help simulate black holes

AI: Surgeon's Assistant or Commodity on a Meter?

Show HN: Hanzi Popup – Chinese Language Reader for iOS

Effective Note Taking

SchemaFlow: Agentic Database Change Impact Analysis, SQL Gen and Eval Guardrails

Making FlashAttention-4 faster for inference

A Missing Woman from Germany Reappears in the Epstein Files

Consciousness likely not unique to earthlings, paper says

As Donald Trump turns 80, he faces a foe he can never defeat: Father Time

An O(x)Caml book that runs

Tribblix: The retro Illumos distribution

How (and Why) SpaceX Will Colonize Mars

Digg Reborn

Story of How Im Running an Unlimited $6/Month AI Provider on 4x RTX 3090s

World Models and the Emergence of a "First-Person" Perspective in an AI [video]

Frontier AI companies will never exceed the capability frontier again

OpenAI hit with multistate probe into possible user harm as its IPO looms

AP Transit – A lightweight 3D real-time NYC subway and PATH map

Git merges can be better

New Documents Detail Nine-Figure, Silicon Valley–Funded "Abundance Movement"

Extinction-Level Capitalism

Phoenix LiveView 1.2 Released

Show HN: Motplot is a crossword but it plays like Sudoku

Quick: An internal hosting platform for the AI era

Forked TensorZero after it was archived after raising $7.3M

Show HN: Raw Binary Program Analysis Tool