We read a lot about the imminent possibility of great harm to, or even the extinction of, humankind - or at least the internet as we know it. Funnily, a lot of these warnings are coming from the creators of the biggest AI services.

Let's put aside for now the obvious, probable interpretation that all of this is just a strategic stunt (PR, market manipulation, you name it...). What would realistic scenarios actually look like?

I've read about abstract possibilities such as terrorist attacks (but more powerful), or AI accidentally damaging infrastructure as soon as we are no longer capable of regaining control (as in the recent Hugging Face incident, but bigger, more frequent, or initiated by a "bad" actor).

With a bit of imagination, I can also think of lots of sci-fi scenarios (like in Daniel Suarez' Daemon). After all, humanity has never had to deal with entities with way greater cognitive capabilities (e.g. imagine social engineering carried out autonomously by AI).

What scenarios do we actually have to fear that go beyond "we don't know, but a lot could happen soon"? What story will we read about human extinction in 10 years (ignoring the fact that we wouldn't be able to write or read it anymore by then)?