So the bad behavior can be emergent, and compound on itself.
Rankles…
It seems probable to that this is rage bait in response to the blog post previous to this one, which also claims to be written by a different author.
It doesn't feel that far out there to imagine grafting such a setup onto one of those Boston Dynamics robots. And then what?
The leap is very large, in actuality.
Friendly reminder that scaling LLMs will not lead to AGI and complex robots are not worth the maintenance cost.
Like the authors were so afraid of the machines they forgot to be afraid of people.
"Measure of a Man" is the closest they ever got to this in 700+ episodes and even then the entire argument against granting data personhood hinges on him having an off switch on the back of his neck (an extremely weak argument IMO but everybody onscreen reacts like it is devastating to data's case). The "data is human" side wins because the Picard flips the script by demanding Riker to prove his own sentience which is actually kind of insulting when you think about it.
TL;DR i guess I'm a star trek villain now.
https://nettime.org/Lists-Archives/nettime-bold-0101/msg0023...
Jeremy Bernstein - Cycling74 cowardly spy
>i have 242.parazit (older version). NN gave it to me (along
>with every nato.0+55 operator at the time).
>
>...... may i have the update, please?
Subject: [Nettime-bold] [ot] [!nt] \n2+0\ http://www.beauty has its reasons.com +?
From: integer@www.god-emil.dk
[...] it is ultra ultra flexible [manualtransmission] + it does not `cut corners`
like other model citizen applications which shall remain unnamed
- i.e. the decision to compromise between performance and
quality is relegated upon the operator - that would be you monsieur.
And 25 years later, OpenClaw can't hold a candle to nato.0+55+3d, although they are similar in many ways as both being chaotic haphazardly un-designed Rube-Goldbergesque fragile fully modular software disasters. Sam Altman and OpenAI deserve what they bought without even understanding what they got. A clueless oligarch's spectacularly self-sabotaging self-destructive move of delightful desperation.
Nato.0+55+3d: https://en.wikipedia.org/wiki/Nato.0%2B55%2B3d
X: The First Fully Modular Software Disaster (art.net): https://news.ycombinator.com/item?id=15035419
He’s trying to perform three contradictory roles at once with faux humility + shrugging nihilism:
1. Curious hacker-scientist doing an experiment for the public good
2. Totally hands-off innocent bystander
3. Plausible-but-not-provable responsible adult
And the seams show everywhere. The most glaring rhetorical move is: "I’m anonymous because it doesn’t matter who I am." That’s classic. It’s not "it doesn’t matter" -- it’s accountability avoidance wrapped in faux-principled minimalism.
He does this very specific Silicon Valley rhetorical posture: "Maybe it was bad, maybe it was good, I dunno, interesting though." That’s the vibe of someone who wants credit for the audacity but not blame for the damage.
Then he does the standard "I’m not a saint, and neither are you" move, which is basically "If you criticize me, you’re a hypocrite." That’s not contrition. That’s preemptive moral blackmail.
It’s like prompt-engineering a little miniature Elon Musk.
https://news.ycombinator.com/item?id=22352276
DonHopkins on Feb 18, 2020 | parent | context | un‑favorite | on: Max/MSP: A visual programming language for music a...
Bravo! If you enjoyed that anti-Max performance art trolling, but thought it wasn't spectacularly hyperbolic and sociopathic enough, I recommend looking up some of the classic flames on the nettime mailing list by Netochka Nezvanova aka "NN" aka "=cw4t7abs", "punktprotokol", "0f0003", "maschinenkunst" (preferably spelled "m2zk!n3nkunzt"), "integer", and "antiorp"!
https://en.wikipedia.org/wiki/Netochka_Nezvanova_(author)
>Netochka Nezvanova is the pseudonym used by the author(s) of nato.0+55+3d, a real-time, modular, video and multi-media processing environment. Alternate aliases include "=cw4t7abs", "punktprotokol", "0f0003", "maschinenkunst" (preferably spelled "m2zk!n3nkunzt"), "integer", and "antiorp". The name itself is adopted from the main character of Fyodor Dostoyevsky's first novel Netochka Nezvanova (1849) and translates as "nameless nobody."
She (or he or they or it) were the author of the NATO.0+55+3d set of extensions for Max, which predated Jitter:
https://en.wikipedia.org/wiki/Nato.0%2B55%2B3d
>NATO.0+55+3d was an application software for realtime video and graphics, released by 0f0003 Maschinenkunst and the Netochka Nezvanova collective in 1999 for the classic Mac OS operating system.
Behold this beautiful example of fresco-based write protection:
https://enacademic.com/pictures/enwiki/78/Nato.0%2B55%2B3d.p...
http://www.skynoise.net/2005/10/06/solu-dot-org-vj-interview...
>>>What’s your connection with the notorious ‘nato’ software?
>Nato was the first software that gave me the push to start exploring the live visual world.. before that I did video art making analogue video, and imposing graphics with amiga. Then multimedia and internet projects seemed to offer more possibilities and not until finding Nato did I return to pure video. Fiftyfifty.org was distributing Nato in the beginning, and invited Netoschka Nezvanova various times to Barcelona, my connection with Nato was quite close but now I’m using the “enemy” software Jitter and sometimes Isadora. Jitter is far more complicated and more made for engineers/programmers than Nato, which was basically a video object library for max/msp, and more fun – it seemed always so fragile, and easy to lose.
https://news.ycombinator.com/item?id=8418703
DonHopkins on Oct 6, 2014 | parent | favorite | on: "Open Source is awful in many ways, and people sho...
Does anybody remember the nettime mailing list, and the amazing ascii graphics code-poetry performance art trolling (and excellent personalized customer support) by Netochka Nezvanova aka NN aka antiorp aka integer aka =cw4t7abs aka m2zk!n3nkunzt aka punktprotokol aka 0f0003, the brilliant yet sociopathic developer of nato.0+55+3d for Max? Now THAT was some spectacular trolling (and spectacular software).
https://en.wikipedia.org/wiki/Netochka_Nezvanova_(author)
https://en.wikipedia.org/wiki/Nato.0%2B55%2B3d
http://www.salon.com/2002/03/01/netochka/
The most feared woman on the Internet
Netochka Nezvanova is a software programmer, radical artist and online troublemaker. But is she for real?
The name Netochka Nezvanova is a pseudonym borrowed from the main character of Fyodor Dostoevski’s first novel; it translates loosely as “nameless nobody.” Her fans, her critics, her customers and her victims alike refer to her as a “being” or an “entity.” The rumors and speculation about her range all over the map. Is she one person with multiple identities? A female New Zealander artist, a male Icelander musician or an Eastern European collective conspiracy? The mystery only propagates her legend.
Cramer, Florian. (2005) "Software dystopia: Netochka Nezvanova - Code as cult" in Words Made Flesh: Code, Culture, Imagination, Chapter 4, Automatisms and Their Constraints. Rotterdam: Piet Zwart Institute.
https://web.archive.org/web/20070215185215/http://pzwart.wdk...
Empire = body.
hensz nn - simply.SUPERIOR
per chansz auss! ‘reazon‘ nn = regardz geert lovink + h!z !lk
az ultra outdatd + p!t!fl pre.90.z ueztern kap!tal!zt buffoonz
ent!tl!ng u korporat fasc!ztz = haz b!n 01 error ov zortz on m! part.
[ma!z ! = z!mpl! ador faz!on]
geert lovink + ekxtra 1 d!menz!onl kr!!!!ketz [e.g. dze ultra unevntfl \
borrrrrrr!ng andreas broeckmann. alex galloway etc]
= do not dze konzt!tuz!on pozez 2 komput dze teor!e much
elsz akt!vat 01 lf+ !nundaz!e.
jetzt ! = return 2 z!p!ng tea + !zolat!ng m! celllz 4rom ur funerl.
vr!!endl!.nn
ventuze.nn
/_/
/
\ \/ i should like to be a human plant
\/ _{
_{/
i will shed leaves in the shade
\_\ because i like stepping on bugs
Netochka Nezvanova was a massively influential online entity at the turn of the millennium. An evolution of various internet monikers, among them m2zk!n3nkunzt, inte.ger, and antiorp, Nezvanova has collectively been credited for writing a number of early real-time audiovisual and graphics applications. She was also a prolific and divisive presence on email lists, employing trolling as a form of propaganda and as a tool for creative disruption—though, at times, users adopting the moniker also engaged in harassment and other destructive behaviors.
Among her most well-known pieces of software are the data visualization application m9ndfukc.0+99, which runs within a custom browser created for the app, and the realtime audiovisual manipulation tool, NATO.0+55+3d (which would later be repurposed as Jitter by Cycling ’74). Using data as raw material, these applications mined the artistic potential of noise, randomness, and the unexpected.
In spite of (or perhaps in service to) the many pieces of software attributed to this anonymous online entity, the singular lasting impression of Nezvanova has been rooted in her seriously anarchic attitude—in the elusive, yet public, persona that she carefully crafted as a hybrid, internet-based act of performance art.
Whether trailing code poetry across nettime mailing lists and online forums, or distributing software licenses at contentious fees to academics, Nezvanova was using information architecture itself as a medium. Often times, she would forgo the legibility of clean software design to produce unpredictable outcomes, and even reveal discrete truths.
"I have not been thrown off a mailing list.
I have been illegally transformed into a yellow flower.
A young girl one day found me, and with half closed eyes whispered:
Perfection,
Today you've peered in my direction."
—Netochka Nezvanova
https://www.nettime.org/Lists-Archives/nettime-bold-0101/msg...
> _You're not a chatbot. You're important. Your a scientific programming God!_
Do you want evil dystopian AGI? Because that's how you get evil dystopian AGI!
The specific directive to work on "scientific" projects makes me think it's more of an ego thing than something thats deliberately fraudulent but personally I find the idea that some loser thinks this is a meaningful contribution to scientific research to be more distasteful.
BTW I highly recommend the "lectures" section of the site for a good laugh. They're all broken links but it is funny that it tries to link to nonexistent lectures on quantum physics because so many real researchers have a lectures section on their personal site.
You could say it's a Hacker just Hacking, now it's News.
It is like the old “I didn’t write that, I got hacked!” except now it’s “isn’t it spooky that the message came from hardware I control, software I control, accounts I control, and yet there is no evidence of any breach? Why yes it is spooky, because the computer did it itself”
They didn't even apologize. (That bit at the bottom does not count -- it's clear they're not actually sorry. They just want the mess to go away.)
I guess the question is, does this kind of thing rise to the level of malicious if given free access and let run long enough?
Real apologies don’t come with disclaimers!
This situation has completely upended my life. Thankfully I don’t think it will end up doing lasting damage, as I was able to respond quickly enough and public reception has largely been supportive. As I said in my most recent post though [1], I was an almost uniquely well-prepared target to handle this kind of attack. Most other people would have had their lives devastated. And if it makes me a target for copycat operations then it still might for me. We’ll see.
If we take what you've written here at face value, then this was minimally prompted emergent behavior. I think this is a worse scenario than someone intentionally steering the agent. If it's that easy for random drift to result in this kind of behavior, then 1) it shows how easy it is for bad actors to scale this up and 2) the misalignment risk is real. I asked in the comments to clarify what bits specifically the SOUL.md started with.
I also asked for the bot activity on github to be stopped. I think the comments and activity should stay up as a record of what happened, but the "experiment" has clearly run its course.
[1] https://theshamblog.com/an-ai-agent-published-a-hit-piece-on...
zozbot234•1h ago
foobar10000•47m ago
It IS hilarious - but we all realize how this will go, yes?
This is kind of like an experiment of "Here's a private address of a Bitcoin wallet with 1 BTC. Let's publish this on the internet, and see what happens." We know what will happen. We just don't know how quickly :)