Netflix Simplified Batch Compute with Kueue

https://netflixtechblog.com/how-netflix-simplified-batch-compute-with-kueue-87860682629c

53•dalvrosa•3d ago

Comments

__turbobrew__•1d ago

Anyone know if Netflix does anything for the k8s storage layer? I imagine they are at the scale where etcd starts to go kaboom? Or maybe they have enough cells where that isn’t a problem?

Given Amazon and Google have their own secret sauce for replacing etcd, I am wondering if Netflix does anything special?

stackskipton•1d ago

It's possible they are using kine: https://github.com/k3s-io/kine

scripni•1d ago

This runs on AWS managed EKS these days, this talk goes into more detail about Netflix's special sauce around the k8s control plane: https://www.youtube.com/watch?v=vaTOiXR2KSM

Netflix actually has much fewer cells than you'd expect btw, their special sauce IMO is federation and using a small subset of k8s APIs.

__turbobrew__•13h ago

I am surprised a company at that scale is running on managed EKS, maybe I underestimate how large the clusters are.

zbentley•1h ago

EKS can get pretty damn big, well into the thousands of nodes without much special tuning, and beyond that with some care and control plane monitoring. Expensive, though.

scripni•1d ago

Congrats, this is awesome!

lukax•23h ago

It's refreshing to see a tech article that isn't about AI. It feels like 5 years ago.

whinvik•20h ago

I see Netflix pumping out tech articles but can't help but notice how much worse the UI experience is getting. Video erroring out, general slowness etc.

Did they just give up?

jamesblonde•18h ago

It certainly feels like Netflix is now a k8s shop. And it probably only a matter of time until they start repatriating workloads to optimize for costs. Then the world will sit up and notice.

beng-nl•15h ago

I don’t get what you’re implying. What is repatriating; You think they will move their workloads to on-prem?

Is there something different about the world that changed the trade-off calculus for cloud vs on-prem from how it was in the last 15 years compared to now?

(I’m as anti-cloud-overspend as the next guy on hn btw. Just trying to make sense of your comment’s worldview.)

jamesblonde•5h ago

Yes, coding agents have reduced the skills/knowledge required to operate workloads on virtualized hardware. K8S and its ecosystem has changed so that it now provides 90% of what you need from the public cloud providers. Big changes that make 8-15X savings by running your own workloads. I think it will be the big players who move first, as they have most to save and have the resources to make it happen.

Claude Sonnet 5

Claude Code is steganographically marking requests

Supersonic flight returning to US after half-century ban

Google copybara: moving code between repositories

Department of Commerce has lifted export controls on Claude Fable 5 and Mythos 5

Forestiere Underground Gardens

Claude Science

Nano Banana 2 Lite

How does a pull-back car work? Illustrated teardown

From brain waves to words: a new path to communication without surgery

Leanstral 1.5

CERN bids farewell to the LHC and enters Long Shutdown 3

Segmenting Robot Video into Actionable Subtasks

I ported Kubernetes to the browser

Hatari – Online Atari ST/STE/TT/Falcon Emulator

Ante: A new way to blend borrow checking and reference counting

Deriving the SVD (Single Value Decomposition) from scratch

Scaling Laws, Carefully

I built a mmWave material classification radar (2025)

Tokyo has only two barley tea makers, we visited one to see how mugicha is made

TabFM: A zero-shot foundation model for tabular data

Stroustrup's Rule (2024)

Building a custom octocopter from scratch with no prior hardware experience

Structural Correctness

Long Island's decommissioned nuclear power plant

Have you restarted your computer this week?

Knoppix

Americans see their country's past, present and future

Reading the internals of Postgres: Database cluster, databases, and tables

Hengefinder