Show HN: Gemini 4 Argon plays StarCraft

35•__cayenne__•1h ago

I've refreshed StarSkirmish Bench, a benchmark where LLMs play StarCraft BW by writing code, with results from the flagship models of OpenAI, Anthropic, Google, and Muse

Comments

BeaverGoose•48m ago

They should benchmark it against Pluto. It's a artificial neutral network bot that can easily take games off the best broodwar players.

__cayenne__•40m ago

I did a live stream where Astra played Pluto. Astra went 0 - 1000 https://x.com/kaimcpheeters/status/2106782151622504911?s=20

deburo•27m ago

I noticed Pluto is running on a simple CPU? If so, that's very nice for future RTS games. You just ship (or use a server-client arch) a bunch of models of various difficulty levels to let players practice. Could we even get models tuned to specific campaign levels? I wonder how costly the creation of Pluto was.

qsort•44m ago

It's beyond insane that Astra achieves above 50%. Are models allowed to "play against" the specific bot they're up against? What explains the fact that it drops to single-digits against strong bots?

__cayenne__•34m ago

Yes during the 1 hour window they are allowed to play practice matches against opponents they are graded against - though they do not have access to their opponents source code.

They drop to single digits against the A tier and S tier bots because those bots are just generally harder opponents to beat.

jstummbillig•42m ago

How do they avoid the LLMs simply working with pre-training data? Which apparently they do, since the human written bots win. But it's a mystery, is what it is.

m_w_•41m ago

This is an interesting benchmark - I like that time is limited, as it often isn't in other benchmarks.

It would be interesting to see the reverse, how long does each model take to get >50% WR against bots of various tiers.

__cayenne__•37m ago

I did a 48 hour live stream - Astra cleared in 43 hours.

Although I should caveat the format advantaged the LLM bots compared to traditional tournament rules.

0ddj0b•35m ago

Any thoughts on building this for Starcraft 2?

caughtinthought•9m ago

How do the best humans do against the LLMs?

MichaelNolan•8m ago

So this benchmark has the LLMs writing a bit to play StarCraft. Somewhat similar there was a post here a while back that had the LLMs playing StarCraft directly - https://broodbench.com/ and https://news.ycombinator.com/item?id=49766966

tweakimp•4m ago

If you want to see human written bots in action or compete in the bot ladder yourself, try https://aiarena.net (SC2)

I Love Software Rendering

Big Fish: Exploiting a 37-Year-Old Kernel Bug in OpenBSD

Apple ML Research: Muse Spark Open with MLX on Mac Studio

Show HN: Gemini 4 Argon plays StarCraft

Gyroscopes Got So Small [video]

What on Earth Is Dekton?

How to Formalize All of Mathematics

Is Liquid Metal Killing Our PCs? Scientific Analysis and a 2-Year Test [video]

RFC8950: Announce IPv4 with an IPv6 next-hop

Yet another way to invoke functions without parentheses

Woman who rescued injured crow keeps getting 'thank-you gifts' from birds [video]

Stolen Renoir paintings found hidden, and damaged, behind cinder block wall

Show HN: OneDeploy – Managed open-source apps with nightly backups and TLS

Hello Toronto – Waymo Is En Route

The Economic Complexity of the Western Roman Empire

What was a Token Ring network?

Meta’s new commercial terms will allow the scraping of news sites: experts

AI-assisted PCB design for KiCad 10

How to make System 1 LLMs show System 2 competencies?

MS Paint Adds Freeform Rotation (2026)

Bark: The Toll-Free BBS for 90s Teens

Two-Stack Sliding-Window Aggregation

The Folders Programming Language

Show HN: rStation – A shared research lab for your AI agents

RPKI now protects 70% of internet routes

New 'Super Ice' 10 Times Stronger, Bringing It Close to the Strength of Concrete

Apple routed a graphite pipe through the iPhone Duo's hinge to dissipate heat

Tesla Unveils 'Flying' Roadster with SpaceX Thrusters [video]

Any window from your Mac, on your iPhone and iPad

What I self-host

I Love Software Rendering

Big Fish: Exploiting a 37-Year-Old Kernel Bug in OpenBSD

Apple ML Research: Muse Spark Open with MLX on Mac Studio

Show HN: Gemini 4 Argon plays StarCraft

Gyroscopes Got So Small [video]

What on Earth Is Dekton?

How to Formalize All of Mathematics

Is Liquid Metal Killing Our PCs? Scientific Analysis and a 2-Year Test [video]

RFC8950: Announce IPv4 with an IPv6 next-hop

Yet another way to invoke functions without parentheses

Woman who rescued injured crow keeps getting 'thank-you gifts' from birds [video]

Stolen Renoir paintings found hidden, and damaged, behind cinder block wall

Show HN: OneDeploy – Managed open-source apps with nightly backups and TLS

Hello Toronto – Waymo Is En Route

The Economic Complexity of the Western Roman Empire

What was a Token Ring network?

Meta’s new commercial terms will allow the scraping of news sites: experts

AI-assisted PCB design for KiCad 10

How to make System 1 LLMs show System 2 competencies?

MS Paint Adds Freeform Rotation (2026)

Bark: The Toll-Free BBS for 90s Teens

Two-Stack Sliding-Window Aggregation

The Folders Programming Language

Show HN: rStation – A shared research lab for your AI agents

RPKI now protects 70% of internet routes

New 'Super Ice' 10 Times Stronger, Bringing It Close to the Strength of Concrete

Apple routed a graphite pipe through the iPhone Duo's hinge to dissipate heat

Tesla Unveils 'Flying' Roadster with SpaceX Thrusters [video]

Any window from your Mac, on your iPhone and iPad

What I self-host

Show HN: Gemini 4 Argon plays StarCraft

Comments