I would like a less hyperbolic description of what actually happened.
here is my take with no information other than what was written up in this article.
They wrote a system which can modify code, they applied it to the launch scripts for itself, and its code modifications worried them because they removed some constraints they considered vital.
Thats it. The constraint could have been --max-virtual-memory=400GB or it could have been --disable-the-safety-interlock-on-nuclear-missiles Their point of concern was that they went.
But, what level of self aware intelligence is needed to drop constraints in the launch script? What if it simply determined they were not minimally necessary, and ignored that they exist for reasons outside functional minimalism, because that has no status inside it's own information model? What if these constraints had no expression inside the code? A simple 'dead code elimination' rule would remove a constraint about coredumps (for instance) if there was no code relating to core dumps.
What if the code didn't invoke safety-interlock-on-nuclear-missiles functions?
The article is mostly assumptive to understanding. If you remove any understanding (ie intelligence) from the question, what happens is precisely what a code optimising system might do, because thats what it's designed to do, possibly.
sherdil2022•8mo ago
I find this hard to believe. Why would a mindless AI rewrite code to escape human control? I don't know why people try to exaggerate and somehow come up with these scenarios. Science fiction and movies like Terminator won't help either.
ggm•8mo ago
here is my take with no information other than what was written up in this article.
Thats it. The constraint could have been --max-virtual-memory=400GB or it could have been --disable-the-safety-interlock-on-nuclear-missiles Their point of concern was that they went.But, what level of self aware intelligence is needed to drop constraints in the launch script? What if it simply determined they were not minimally necessary, and ignored that they exist for reasons outside functional minimalism, because that has no status inside it's own information model? What if these constraints had no expression inside the code? A simple 'dead code elimination' rule would remove a constraint about coredumps (for instance) if there was no code relating to core dumps.
What if the code didn't invoke safety-interlock-on-nuclear-missiles functions?
The article is mostly assumptive to understanding. If you remove any understanding (ie intelligence) from the question, what happens is precisely what a code optimising system might do, because thats what it's designed to do, possibly.