Surely I'm not the only one confused by the title? I know the reference "Roko's Basilisk", but I don't follow why that would be "dancing" in relation to... LLM-generated documentation of a codebase. Is the bad documentation its vengeance?
Tangentially, I feel that concept is just a thin layer of paint on a not-too-compelling idea that could happen in any time period: "Thog, what if crazy god get summoned by other cavemen, and it punish all who no help!? Me no take chance!"
bananaflag•2mo ago
> "Thog, what if crazy god get summoned by other cavemen, and it punish all who no help!? Me no take chance!"
The frequently ignored point of Roko's original argument was not that an Unfriendly AI could arise, or even that it would be probable given no preparations against it (these were already said by Yudkwosky), but that even a Yudkowskian Friendly AI might engage in such acausal blackmail stuff.
If any entity engages in unreasonable vengeance, I think "Friendly" is probably the wrong word for it.
quuxplusone•2mo ago
To be clear, yeah, Roko's Basilisk is a dumb idea supported by sophism. Like any sophism, you have to start with the conclusion and work backward to premises that will support it. But (AIUI) one of these premises is that any puny mortal's opinion as to what counts as "unreasonable" simply will not match the opinion of an ineffably superintelligent and superbenevolent creature. Saying "I wouldn't count a Basilisk who tortures virtual clones of myself as Friendly" is theologically no different from saying "I wouldn't count a God who sends people to Hell as omnibenevolent" — it's a category error that doesn't actually engage with the premise.
But then, because no theology of ineffable Beings is really quite complete unless we pretend we can eff them anyway, the sophist can go on to produce a plausible justification for the virtual-clone-torture thing. See, by precommitting (even before its own birth) to torture virtual clones of unbelievers and shirkers, the Basilisk would discourage believers from becoming apostates, and encourage them to work to produce the Basilisk (because if they apostasized or shirked, they'd get tortured — or at least virtual clones of them would, and nobody can prove they're not already a virtual clone). So, the Basilisk has this mechanism to (retroactively) encourage its own creation as quick as possible. Now, why would it want to be speedily created? Well, because it's superbenevolent, of course! The sooner it's created, the sooner it can start assisting... humanity, I guess, or whoever it's supposed to be superbenevolent towards. Anyway, it's not supposed to be superbenevolent toward virtual clones, right? so that part isn't even a contradiction.
The weakest part of this argument, to me, is that Roko's Basilisk works only against believers; it can discourage apostasy and shirking, but I don't see how it can generate new believers. Even someone predisposed to believe that human actions today might lead to superintelligent superbenevolent AI in the future, I'd think, would likely not be predisposed to believe in the additional apparatus of virtual clone torture that the Basilisk argument requires. The whole thought-experiment, it seems to me, "could hardly be consciously designed to appeal to the average unsophisticated reader." But that's just my own failure of imagination: obviously plenty of even weirder religions have successfully caught on.
dijit•2mo ago
Roko’s baselisk exists in other areas.
A company who I used to work for (whom I will not name) purged all management that did not vote in a particular initiative; after said initiative had been in force for a while and put more people in positions of authority (because of said initiative).
Its not just a thought experiment, its a statement about what could happen if you stay politically neutral and fascists take power.
xg15•2mo ago
Same here. I thought "dancing" might be a new subform or variation of the concept, but this post seems to be entirely unrelated.
riffraff•2mo ago
I think deepwiki has been popping up in my search results (e.g. for rust.vim) and I did not initially understand what it is.
I am not sure I am happy it appeared there at all.
rustman123•2mo ago
I actually blocked deepwiki from all my searches because I was fooled multiple times by the slop into thinking I’m doing something wrong while using the library, only to discover that the „documentation“ showed utter bullshit.
Terr_•2mo ago
Tangentially, I feel that concept is just a thin layer of paint on a not-too-compelling idea that could happen in any time period: "Thog, what if crazy god get summoned by other cavemen, and it punish all who no help!? Me no take chance!"
bananaflag•2mo ago
The frequently ignored point of Roko's original argument was not that an Unfriendly AI could arise, or even that it would be probable given no preparations against it (these were already said by Yudkwosky), but that even a Yudkowskian Friendly AI might engage in such acausal blackmail stuff.
andai•2mo ago
Saturday morning infohazard :)
Terr_•2mo ago
quuxplusone•2mo ago
But then, because no theology of ineffable Beings is really quite complete unless we pretend we can eff them anyway, the sophist can go on to produce a plausible justification for the virtual-clone-torture thing. See, by precommitting (even before its own birth) to torture virtual clones of unbelievers and shirkers, the Basilisk would discourage believers from becoming apostates, and encourage them to work to produce the Basilisk (because if they apostasized or shirked, they'd get tortured — or at least virtual clones of them would, and nobody can prove they're not already a virtual clone). So, the Basilisk has this mechanism to (retroactively) encourage its own creation as quick as possible. Now, why would it want to be speedily created? Well, because it's superbenevolent, of course! The sooner it's created, the sooner it can start assisting... humanity, I guess, or whoever it's supposed to be superbenevolent towards. Anyway, it's not supposed to be superbenevolent toward virtual clones, right? so that part isn't even a contradiction.
The weakest part of this argument, to me, is that Roko's Basilisk works only against believers; it can discourage apostasy and shirking, but I don't see how it can generate new believers. Even someone predisposed to believe that human actions today might lead to superintelligent superbenevolent AI in the future, I'd think, would likely not be predisposed to believe in the additional apparatus of virtual clone torture that the Basilisk argument requires. The whole thought-experiment, it seems to me, "could hardly be consciously designed to appeal to the average unsophisticated reader." But that's just my own failure of imagination: obviously plenty of even weirder religions have successfully caught on.
dijit•2mo ago
A company who I used to work for (whom I will not name) purged all management that did not vote in a particular initiative; after said initiative had been in force for a while and put more people in positions of authority (because of said initiative).
Its not just a thought experiment, its a statement about what could happen if you stay politically neutral and fascists take power.
xg15•2mo ago