The paper is very accessible (it's mostly written by Anthropic researchers), and Section 4 summarises their findings really well. They were themselves really surprised by the results:

> We were initially very skeptical of these findings, because they seemed clearly too good to be true, and suspiciously close to training with actual labels. To ensure we didn’t accidentally train on the labels, (1) we re-ran the experiment several times on different datasets, (2) we copied the dataset into a new file, excluding any labels before re-running our algorithm with that file, and (3) one coauthor independently replicated the findings on the Claude 3.5 Haiku base model using a different codebase.

(emphasis mine)

Techniques can be arbitrarily old & common in industry, but still be a novel academic paper, first to document & evaluate key aspects in that separate (& often lagging) canon.

My immediate thought is how this differs from just naively asking each each question individually to the LLM multiple times and taking the ground truth as consensus majority. The search algorithm probably implicitly does this, though I guess there is some benefit in providing it other related questions as well. I thought I remember similar papers dealing with LLM self-interrogation, using the idea that "true" statements must be logically consistent and so the same underlying explanation should hold for perturbations or related questions as well.

The flaw seems to be that it's still beholden to the training data. Any misconceptions that are internalized during pretraining won't actually be fixed, and in fact they'll only be propagated further.

The term is often used in fiction, particularly in superhero comics and fantasy, but it can also be used metaphorically to describe extraordinary effort or achievement in real life (e.g., "It took a superhuman effort to finish the marathon").

(Definition from Gemini)

It seems reasonable to use the term to me simply to say the abilities on a benchmark of the model were greater than the human annotated data. Computers have always been superhuman at many tasks, even before llms.

Math problems: Two solutions to the same problem can't both be "True" if they have different final answers

Comparisons: "A > B" and "B > A" can't both be "True" (asymmetry constraint)

The function returns 1 for inconsistencies, 0 for consistent pairs. But the paper doesn't explain how to detect "same math problem" or implement the asymmetry check in practice. From the implementation hints, it seems they use simple pattern matching - probably checking if question text is identical for math problems, and detecting comparison pairs through linguistic patterns. The authors explicitly say they use "simple and general logical constraints" rather than fine-grained consistency checking.

This is one of those critical implementation details that makes reproduction difficult. You'd likely need to implement domain-specific heuristics: exact string matching for duplicate problems, regex patterns for comparisons, maybe basic semantic similarity for near-duplicates. The function seems designed to prevent obvious degenerate solutions (labeling everything the same) rather than enforce comprehensive logical consistency.

It's a significant gap in an otherwise interesting paper - the consistency term appears crucial for preventing the algorithm from gaming the mutual predictability objective, but they don't give enough detail to actually implement it reliably.

  def generate_consistency_tests(examples, task_description):
    prompt = f"""
    Given this task: {task_description}
    And these example pairs: {examples[:5]}
    
    Write Python functions to detect when two examples should have:
    1. The same label (duplicates, equivalent problems)
    2. Different labels (contradictions, opposites)
    3. Asymmetric constraints (A>B excludes B>A)
    
    Return executable code for consistency_check(xi, yi, xj, yj).
    """
    
    generated_code = llm.generate(prompt)
    return compile_and_validate(generated_code)