Related ongoing thread: Chain of thought monitorability: A new and fragile opportunity for AI safety - https://news.ycombinator.com/item?id=44582855

Watch (or read) The 3 Body Problem[1]. If you substitute AI (artificial intelligence) for AI (alien intelligence, specifically the San Ti), then it could give us a roadmap for what to expect, including the fact that there are likely going to be humans who invite or specifically enable AI to supersede humans.

If AI vendors are truly concerned about the problem as expressed here, I would advance that it is a poor strategy to thinking "we have a limited transparency window now" and bank on that; a much smarter strategy would be to assume a more adversarial dynamic and assume that AI is already actively hiding intentions from us, manipulating us to grow comfortable. Only the paranoid will survive. An alternative position would suggest that the AI vendors don't fully believe the risk is real.

[1] https://www.domusweb.it/en/news/2024/04/10/in-3-body-problem...