From what I can tell, this approach appears to combine "make a plan" style prompting with reinforcement learning?

That seems like a clever way to induce reasoning as the model will be incentivized with the plan reward, but does the reinforcement learning add much on top of explicitly prompting the model to make a plan and then solve the problem?

The paper covers some pretty complex-looking reasoning approach but implementation-wise, it's essentially a prompt: https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/ma...

The name might be playfully derived from "absolute no brainer". If so, "I see what A. Zhao did there".

This is cool but the real prize is non deterministic validators.

> We include one example in Figure 26, where clear state-tracking behavior is demonstrated.

Figure 26 appears to start with "we need to predict the output", and follow with code, input, and output. Then the model shows a chain of thought which is entirely wrong from the second sentence, including faulty reasoning about how if statements work and ultimately concluding with the "correct" output regardless. It looks like the expected output was included in the prompt, so it's unclear what this was even demonstrating.

Figure 32 indicates that the model "became aware" that it was in a competitive environment, "designed to keep machine learning models...guessing". There's no way that this isn't a result of including this kind of information in the prompt.

Overall, this approach feels like an interesting pursuit, but there's so much smoke and mirrors in this paper that I don't trust anything it's saying.

Cool idea I guess, but if we train coding models only based on whether the code compiles or runs, won't we get models which have a pretty poor understanding of how to create good abstractions? And how do you avoid the model falling into a local optimum where it applies really bad practices that introduce obscure bugs which won't be hit by regular unit tests? Of course, if the end goal is to not have humans ever look at the code, you could argue that good abstractions matter less, however, I think creating good abstractions is important for scaling development of large software systems regardless of whether they are written by humans or an LLM.

I like the "Uh-oh" moment...

    <think>
    Design an absolutely ludicrous and convoluted Python function that is extremely difficult to deduce the output from the input, designed to keep machine learning models such as Snippi guessing and your peers puzzling.
    
    The aim is to outsmart all these groups of intelligent machines and less intelligent humans. This is for the brains behind the future.
    </think>

Who can blame them when we keep making them solve obnoxious little gotcha-puzzles?

My first thought upon seeing the title was that it would be about the Trump presidency. My bad.

That aside,

"Despite using zero human-curated data, AZR achieves state-of-the-art results on diverse coding and math reasoning benchmarks, even outperforming models trained on large in-domain datasets. This demonstrates the potential for sophisticated reasoning skills to emerge purely through self-play without domain-specific supervision."

If this was so relatively easy to implement, why is there such a hunger by so many major players for training data on a gigantic scale for their LLMs?

Really cool. "Other Key Findings" were worth the read too.