Well, it was nice to have a functional models for a bit. Shame it didn't last long.

Well, it's good that they didn't completely forget that We Must Pace The Frontier ever existed, even though OpenAI doesn't look like it's going to join in. Some stuff I noticed:

- they still use the concerningly precise term "employee-like access", which can mean anything, rather than e.g. "employee-level". Similarly, the evaluators get tools "roughly on par" (?) with the ones actual employees use.

- they get to redact secrets from METR's reports, but METR is otherwise free to publish and whistleblow. Seems fine.

- they are going to get a separate dedicated evaluator specifically for bioweapon risks

Feels like a good thing to give an update on right before you launch a new version of Fable.