One of my side projects is a game, and I've made it so agents can spawn off their own world shard to play in. There's a little Chrome-driven agent harness that can grab low res screenshots while it speaks directly to the game over the wire to control things.

I've had Luna play it and it can drive around (you play as a bike) but it did struggle to play one of the mini-games (Disc Golf). My idea was that model trainers could run it an RL loop using the harness to get feedback data and screenshots to use, then reset the game world as needed during the process.

A smart model capable of multi-modal inference on screenshots/game context at 30-60fps could be pretty interesting, and I'm sure it's coming. No idea how much it would generalize but would be fun to see done.

This formative moment could never happen today, because the wilderness is behind 3 confirmation screens instead of just being something you get to discover in the game:

> You have to know that [items] are worth something in order to not get taken advantage of. And it goes to kind of extreme levels too because not only will people, try to give you a bad price, but the game is just really well known for the scamming and the lying and stealing your password. And it's really the wild west out there.

> And so, and really this is true. My very first memory of the very first time that I entered the world of RuneScape, I was asking somebody for help. I said like I was trying to make bread and I was like, "Where can I find a mill so I can make some bread?" And a guy was like, "Follow me. I'm going to show you." And I followed him like into these woods. And you know, you can imagine he turned around. He started to attack me. And I realized what was happening. I was probably 11 years old. I realized what was happening. And I made a break for it. I ran back south for cover and I got away. I didn't die. And I remember like, you know, closing the game.

> I think that my parents hadn't been home, so I had time to think about this. And I was like, that was the coolest thing that's ever happened to me. And I think that this is kind of the beauty and the magic of this game is that it's a little bit of like a rough and tumble world where you have to look out for yourself. And as a kid, it's like that's actually really great practice to be out in the world with real people and have to understand what's going on and learn things like, oh yeah, if somebody offers you a really great deal, maybe be careful.

Cheers to this kind of game design. Wish we'd see it again some day.