It shows practically how the accumulation of unverified behaviors (or verification without deliberate specification) plays out at a hyper-scale, and how the "verification paradox" turns localized failures into cascading system collapse.
You can read the case study here: https://doi.org/10.5281/zenodo.18980467
I think looking at real-world post-mortems through this lens makes the danger of the AI-driven "E_v trap" much more tangible.
frannyPS•2h ago
I realized our current theory lacks the vocabulary to diagnose why this is happening. The paper proposes a two-axis model (Specification vs. Verification) yielding four categories of software behavior: S_v, S_u, E_v, and E_u.
The most dangerous trap right now is what I call E_v (unspecified-but-verified). AI generates implementation and tests at machine speed. This creates a "verification paradox": organizations accumulate E_v behaviors (tested without specification), creating the appearance of quality (green CI, high coverage). However, the tested behaviors have no basis for evaluating correctness because no deliberate human decision (specification) was made. AI accelerates Axis 1 (does it do what the spec says?) while leaving Axis 2 (does the spec capture what is valuable?) untouched.
I originally submitted this to IEEE Software. It was desk-rejected for being a "conceptual model" rather than a localized case study with metrics. But I wrote this for practitioners, to give us a structural vocabulary for the AI-era mess we are currently in.
I'd love to hear if this E_v trap and the verification paradox resonate with what you're seeing in your own AI-assisted workflows.