AI now writes most of the code at frontier AI developers. Anthropic says Claude leads 26% of its R&D, and staff there and at OpenAI describe losing track of what the models built. Every proposal for pacing self-improvement gates on the AI (capability evals, compute caps, a lag before a model does AI R&D). None gates on whether the people responsible can still explain the work.
In comprehension audits independent auditors are embedded at an AI developer and watch its workstream. They pick a contribution: a meaningful unit of R&D output such as a completed experiment, a new training recipe, or integration of new data. They study the work and prepare, then call a short notice audit meeting with the responsible people. In that meeting they ask questions to test whether the R&D staff understand what the contribution does and how it works, not model internals. The meeting is like a thesis defense or design review, with staff answering without AI assistance and conducted in a blameless manner.
Across 2000+ leading open source AI R&D projects we see human review comments dropped about 50% per LOC from early 2024 to mid 2026.
I'd love feedback on the approach and how developers are keeping track of agentic output.
ronbodkin•29m ago
AI now writes most of the code at frontier AI developers. Anthropic says Claude leads 26% of its R&D, and staff there and at OpenAI describe losing track of what the models built. Every proposal for pacing self-improvement gates on the AI (capability evals, compute caps, a lag before a model does AI R&D). None gates on whether the people responsible can still explain the work. In comprehension audits independent auditors are embedded at an AI developer and watch its workstream. They pick a contribution: a meaningful unit of R&D output such as a completed experiment, a new training recipe, or integration of new data. They study the work and prepare, then call a short notice audit meeting with the responsible people. In that meeting they ask questions to test whether the R&D staff understand what the contribution does and how it works, not model internals. The meeting is like a thesis defense or design review, with staff answering without AI assistance and conducted in a blameless manner. Across 2000+ leading open source AI R&D projects we see human review comments dropped about 50% per LOC from early 2024 to mid 2026. I'd love feedback on the approach and how developers are keeping track of agentic output.