Outcome-Based Reinforcement Learning to Predict the Future

99•bturtel•8mo ago

Comments

ctoth•8mo ago

Do you want paperclips? Because this is how you get paperclips!

Eliminate all agents, all sources of change, all complexity - anything that could introduce unpredictability, and it suddenly becomes far easier to predict the future, no?

JoshTriplett•8mo ago

> Do you want paperclips? Because this is how you get paperclips!

Don't^W worry, there are many other ways of getting paperclips, and we're doing all of them.

sitkack•8mo ago

Even explaining how not to get paper clips, gets you paper clips when you can invert the loss function. Paper clips for everyone!

vlovich123•8mo ago

I don't know. Paperclips are awful useful. Would it be so bad to build more of them?

throwaway71271•8mo ago

https://www.decisionproblem.com/paperclips/index2.html go ahead :)

Ygg2•8mo ago

That's all fun and games until paperclip maximizers starts looking at your blood as source of iron.

valine•8mo ago

So instead of next token prediction its next event prediction. At some point this just loops around and we're back to teaching models to predict the next token in the sequence.

lumost•8mo ago

Tokens are an awfully convenient way to describe an event.

phyalow•8mo ago

Tokens are just discretized state representations.

ww520•8mo ago

It’s the next state. So instead of spitting out words, it will spit out a whole movie, or a sequence of world states in a game or simulation.

amelius•8mo ago

Why would you use RL if you're not going to control the environment, but just predict it?

TOMDM•8mo ago

Because they're training a predictor, not an agent?

jldugger•8mo ago

From the abstract

> A simple trading rule turns this calibration edge into $127 of hypothetical profit versus $92 for o1 (p = 0.037).

I'm lazy: is this hypothetical shooting fish in a barrel, or is it a real edge?

nyrikki•8mo ago

Note the 'hypothetical profit' part , I know of several groups looking for opportunities to skim off LLM traders, leveraging its limited sensitivity, expressiveness, and the loss of tail data.

Predictive AI is problematic no matter what tool you use. Great at demoware that doesn't deliver.

I am sure there are use cases, but it would be augmentation, not a reliable approach by itself.

garbagecoder•8mo ago

"a couple of wavy lines"

bzzzzz "sorry this isn't your lucky day"

We Mourn Our Craft

I Write Games in C (yes, C)

Hoot: Scheme on WebAssembly

SectorC: A C Compiler in 512 bytes

Stories from 25 Years of Software Development

U.S. Jobs Disappear at Fastest January Pace Since Great Recession

OpenCiv3: Open-source, cross-platform reimagining of Civilization III

The AI boom is causing shortages everywhere else

Al Lowe on model trains, funny deaths and working with Disney

The Waymo World Model

Reinforcement Learning from Human Feedback

Brookhaven Lab's RHIC Concludes 25-Year Run with Final Collisions

Start all of your commands with a comma (2009)

Vocal Guide – belt sing without killing yourself

France's homegrown open source online office suite

Coding agents have replaced every framework I used

A Fresh Look at IBM 3270 Information Display System

Show HN: I saw this cool navigation reveal, so I made a simple HTML+CSS version

History and Timeline of the Proco Rat Pedal (2021)

Selection Rather Than Prediction

72M Points of Interest

Unseen Footage of Atari Battlezone Arcade Cabinet Production

Where did all the starships go?

Show HN: Look Ma, No Linux: Shell, App Installer, Vi, Cc on ESP32-S3 / BreezyBox

Learning from context is harder than we thought

Monty: A minimal, secure Python interpreter written in Rust for use by AI

Show HN: Kappal – CLI to Run Docker Compose YML on Kubernetes for Local Dev

Hackers (1995) Animated Experience

Making geo joins faster with H3 indexes

Sheldon Brown's Bicycle Technical Info