This is a reinforcement learning environment I made for post-training large language models that are already post-trained on coding and math. Optimization problems in general are good verifiable reward problems, with lots of intermediary rewards because even relatively simple algorithms can solve very easy instances while superhuman-level algorithms, similar to what OpenAI displayed with the Heuristic Programming Contest yesterday, are the end state after substantial post-training. AES inversion framed as an optimization problem seemed like an interesting problem to formalize like this because we have so much empirical evidence it is a hard problem for many rounds, but it is known to be solvable for very small number of rounds.

The RL env has a generic solver interface that the model must implement and scores the model using an objective function designed to have lots of intermediary rewards. In particular the model scores high for solving a small number of rounds of AES, and also for partially solving large rounds of AES using an internal representation that allows us to measure how many violations a "near" solution has. I would love any feedback or thoughts.