I genuinely like the framing of advisory versus authoritative AI, and I agree with the core observation that authority, when it is genuinely granted, is what unlocks step change improvements rather than marginal efficiency gains. In the environments where it is appropriate, allowing systems to act rather than merely suggest can dramatically accelerate development and reshape workflows in ways that advisory tools never will. In that sense, you are right: authority is the AI bottleneck.
My concern with your article is that, without clearer caveats, you imply that authority is the right answer everywhere. As you rightly note, AI systems make mistakes and they make them frequently. In many real world contexts, those mistakes are not cleanly reversible. You cannot roll back a data leak. You cannot always recover fully from data loss. You cannot always undo millions of pounds of lost or refunded revenue caused by subtle failures or downtime. You cannot always roll back the consequences of an exploited security vulnerability. And you certainly cannot reliably undo reputational damage once trust has been lost.
Even in cases where you can mostly recover from a failure, you cannot recover the organisational and human disruption it causes. A recent UK example is the case where thousands of drivers were wrongly fined for speeding due to a system error that persisted from 2021. Given the scale, some will have lost their licences, some may have lost their jobs, and many will have experienced long term impacts such as higher insurance premiums. Even if fines are refunded or records corrected later, the downstream consequences cannot simply be undone. While the failure in this example was caused by human error, the fact that some mistakes are unrecoverable is just as true for AI.
Part of the current polarisation in opinions about AI comes from a lack of explicit context. People talk past each other because they are optimising for different objectives in different environments, but argue as if they are discussing the same problem. An approach that is transformative in a low risk internal system can be reckless in a public, regulated, or security sensitive one.
Where I strongly agree with you is that authoritative AI can be extremely powerful in the right domains. Proofs of concept are an obvious example, where speed of learning matters more than correctness and the blast radius is intentionally small. Many internal or back office applications fall into the same category. However, for many public facing, safety critical, or highly regulated systems, authority is not simply a cultural or organisational choice. It is a hard constraint shaped by risk, liability, regulation, and irreversibility. In those contexts, using AI in a strictly advisory capacity may be a bottleneck, but it is also a deliberate and necessary control measure, at least for now.
scresswell•1mo ago
My concern with your article is that, without clearer caveats, you imply that authority is the right answer everywhere. As you rightly note, AI systems make mistakes and they make them frequently. In many real world contexts, those mistakes are not cleanly reversible. You cannot roll back a data leak. You cannot always recover fully from data loss. You cannot always undo millions of pounds of lost or refunded revenue caused by subtle failures or downtime. You cannot always roll back the consequences of an exploited security vulnerability. And you certainly cannot reliably undo reputational damage once trust has been lost.
Even in cases where you can mostly recover from a failure, you cannot recover the organisational and human disruption it causes. A recent UK example is the case where thousands of drivers were wrongly fined for speeding due to a system error that persisted from 2021. Given the scale, some will have lost their licences, some may have lost their jobs, and many will have experienced long term impacts such as higher insurance premiums. Even if fines are refunded or records corrected later, the downstream consequences cannot simply be undone. While the failure in this example was caused by human error, the fact that some mistakes are unrecoverable is just as true for AI.
Part of the current polarisation in opinions about AI comes from a lack of explicit context. People talk past each other because they are optimising for different objectives in different environments, but argue as if they are discussing the same problem. An approach that is transformative in a low risk internal system can be reckless in a public, regulated, or security sensitive one.
Where I strongly agree with you is that authoritative AI can be extremely powerful in the right domains. Proofs of concept are an obvious example, where speed of learning matters more than correctness and the blast radius is intentionally small. Many internal or back office applications fall into the same category. However, for many public facing, safety critical, or highly regulated systems, authority is not simply a cultural or organisational choice. It is a hard constraint shaped by risk, liability, regulation, and irreversibility. In those contexts, using AI in a strictly advisory capacity may be a bottleneck, but it is also a deliberate and necessary control measure, at least for now.