> When nuclear fission was discovered in the late 1930s, physicists concluded within months that it could be used to build a bomb. Epidemiologists agree on the potential for a pandemic, and astrophysicists agree on the risk of an asteroid strike. But no such consensus exists regarding the dangers of A.I., even after a decade of vigorous debate.

We have problems because AI breaks existing paradigms.

Historically speaking, we have been able to generate consensus because those things without prior examples were “non-complex” and those that did spawn complexity or arose out of chaos and chance had prior art that demonstrated patterns.

So nuclear bombs could be predicted out of the limited uses of fission, and pandemics arising out of random unpredictable mutations could be predicted out of the copious prior examples of novel lethal viruses running amuck.

But now we have something with no prior examples which generates highly complex possibilities and is itself chaotic and random and not very well understood. And that completely breaks our ability to generate a consensus, because without prior examples we simply cannot tell which of the innumerable branching of possibilities is legitimately possible, and which are dead ends.

Or worse -- distractions that blind us to the dangerous paths it is stumbling down..