For years, the lock-in was dead simple: CUDA.

Want top-tier performance? You wrote CUDA. Do that once, and you were all-in on NVIDIA. The ecosystem compounded—libraries, tooling, docs, talent—everything reinforcing the same gravity well.

That world is starting to crack.

We’re entering a phase where low-level code isn’t a rare skill anymore. Models are capable at generating kernels, bindings, and glue code. Good enough to get a first version running fast and iterate from there. The switching cost to a new accelerator is dropping quickly. What used to require a dedicated team now often looks like a decent prompt plus a few review passes.

There’s an old David Wheeler line: All problems in computer science can be solved by another level of indirection. AI Codegen is an abstraction and more.

At the same time, the economics are shifting.

For many real workloads—especially inference—VRAM matters more than peak FLOPS. You want models resident in memory, batching cleanly, with predictable latency. On a dollars-per-GB basis, AMD is starting to look compelling. Newer cards bring stronger low-precision throughput (FP8/INT4), structured sparsity, and significantly more memory on mainstream SKUs. If you’re running open models and care about cost/throughput, you’re at least evaluating them.

Intel is entering the mix as well. Battlemage (Arc Pro B-series) pushes high VRAM configurations with competitive price/perf for local inference. Not dominant, but another viable option that didn’t exist in the CUDA-only world.

Then there’s supply.

NVIDIA has built enormous demand and maintained pricing power. But scarcity cuts both ways. If you can’t get hardware—or only at extreme prices—people explore alternatives. Startups take what they can get. Infra teams design for heterogeneity. Open source adapts to whatever is available.

This is how moats erode: not via a single replacement, but through many small workarounds that become standard.

Two datapoints from actually standing up a modern model serving stack:

1. In my recent GLM 5.1 deployment on 8xB200s, getting a novel model to serve reliably was painful. It took me ~12–13 minutes of cold starts (many!), random restarts, non-obvious flags, kernel warmups, and graph captures just to reach a stable baseline. Most of that wasn’t “AI”—it was infra whack-a-mole across memory limits, runtimes, and config quirks.

2. Even once it was running, it was fragile. I kept hitting issues like streaming tool calls producing invalid JSON because the model output, server parser, and client SDK were out of sync. Fixing it required patches across multiple layers just to get to consistent outputs. Real systems are leaky—far from clean abstractions.

That’s the actual moat: not CUDA, but the entire stack—libraries, compilers, interconnects, and years of ops knowledge.

But, it’s early.

CUDA isn’t just a platform; it’s a decade-plus of battle-tested infra. Getting something to run is one thing. Getting it to run great at scale is still difficult, performance cliffs in exactly the wrong places.

And NVIDIA is moving up the stack aggressively—higher-level APIs, inference tooling, tighter framework integration. Blackwell-class hardware pushes further efficiency (e.g., low-precision compute like FP4) and targets memory-bound inference directly. If abstractions become the battlefield, they’re positioning to control that layer too.

So what happens:

* Near term: NVIDIA continues to dominate. Demand is still growing fast and they remain the default. * Medium term: the edges fray. Inference becomes more heterogeneous. AMD and Intel pick up share where cost and memory dominate. * Long term: value shifts upward—to models, data, orchestration. Hardware still matters, but becomes more interchangeable at the margin.

Bottom line: CUDA used to be a wall. Now it’s closer to a speed bump. AI didn’t remove the moat—it just made it much easier to cross when there’s a reason to.

DevLens understand codebases faster(onboarding, PR Review,Blast Radius)

Audi A1 e-tron driven (2010)

Relive the lunar flyby and re-entry of Artemis II – Orion Integrity

Show HN: 7 Years of Tropospheric NO₂

Who Uses AI?

Toyota joins hydrogen truck alliance push

Gobo

Our response to the Axios developer tool compromise

Training LLMs to Predict World Events

Show HN: Spuddy.sh – Describe your API in terminal get a hosted API with state

Cyclone Ditwah

Why It's So Hard to Fix an Electric Bike

Show HN: PDF-Proof – Make Claude show where it found each value in a PDF

Gemma4

Nvidia's moat is not what it used to be

Sol-RL: FP4 Explore, BF16 Train

Remove iOS Contact Posters

SuperCmd – Open-source launcher for macOS with Raycast-compatible extensions

India hits 150 GW solar milestone

Music hardware company Elektron acquired by investment company Bonnier Capital

Show HN: Turn 3 letters into a cube with different projections (inspired by GEB)

Agentic Specification Engineering

A Breaking Bad grand strategy browser game – pure JavaScript, no frameworks

Test your CV against any job description – free, open-source, AI-powered

Tesla Disables the FSD Used Illegally in over 100k Cars

Historical Java (2014)

Building the first AI Red Team OS – mythosai.cloud – early access open

Private Equity Captured the Ambulance Market

A private collection of 10k Chicago show tapes finds a public home

Show HN: Millions of websites crawled for LLMs to rebuild the pricing pages

Show HN: Kafkaesque – a wire-compatible mock Kafka

Comments