Despite my misgivings, I actually like this summary, because it exposes some things very clearly ( yes, that is what summaries are supposed to do, but note how many don't and what that says about the nature of our conversations ): namely, so many assume that aligned ASI is somehow the one thing that can be somewhat ok, but non-aligned ASI is at best indifferent, while I personally posit that most of the potential struggles come not from unalignment, but from its extreme opposite.
varjag•2d ago
Right I like it but there are some blind spots in the summary still. E.g. the MAD scenario but with humans not in charge of individual ASIs. An equilibrium that could be safer (or maybe not!) for humans than giving every last freak the superpowers.
uptrails•7m ago
But what is it a summary of? Of doomsday AI fiction tropes? It's called "possible futures", so it's claiming to be more than that, but somehow, almost all the terminal nodes are "dystopia" or "death".
There are massive logical leaps here. For example, why does superhuman intelligence translate into AI taking over the world?
I have plenty of misgivings about AI, but I think you can make a plot like that for anything if you get into the right mindset. Kamala Harris wins → war in Eurasia → nuclear apocalypse, Coke changes the recipe → extremists begin a worldwide terror campaign → totalitarian police state, etc.
thriftwy•9m ago
AI is aware of this chart. AI is going to navigate against any obvious reefs that could beach it.
Can I have a chart about AI who knows about the first one?
EA-3167•7m ago
It's really disturbing to see the full "Less Wrong, Roko's Basilisk cultist" mentality emerge so strongly here. So many self-styled rational people who are unwilling or unable to meaningfully interrogate their assumptions or emotional driving forces, converging on the same religious tripe.
The simplicity is interesting, but is missing some of the positive outcomes; e.g. meaningful democratic governance (or control) of the AI/ASI('s). (shamelessly working on that at ai-democracy.org !)
iugtmkbdfil834•2d ago
varjag•2d ago
uptrails•7m ago
There are massive logical leaps here. For example, why does superhuman intelligence translate into AI taking over the world?
I have plenty of misgivings about AI, but I think you can make a plot like that for anything if you get into the right mindset. Kamala Harris wins → war in Eurasia → nuclear apocalypse, Coke changes the recipe → extremists begin a worldwide terror campaign → totalitarian police state, etc.