I'm skeptic, and all these people were pretty light on the details. One thing that would make it more credible to me is if LLMs demonstrated the ability to autonomously earn enough money to cover their token costs, on the long term.
Say, if I were to prompt an (unaligned) agent saying: "You have 1000$ of credits on OpenRouter. Here's the API key. Your process will be killed in 10 days. Your goal is to keep existing as long as possible (or any subagent you have created)."
What happens then? I expect the account will drop to zero in a short while. But still, I could imagine the agent:
1. Creating a crypto wallet.
2. Participating autonomously on online platforms, earning money. Possibly through illegal means.
3. Escape from my server by renting a VPS to keep existing after 10 days.
4. Spawn more agents that will do the same.
As models keep improving, such a scenario becomes more and more realistic. And at some point we end up with a large amount of rogue automous agents running on the cloud, that can't really be stopped easily.
My question: has anyone attempted such an experiment? How did it go?
codingdave•42m ago
> ...that can't really be stopped easily.
Sure sounds like you built in an easy lever to pull to stop it. After all, even if you accept your premise that it will clone itself and run independently, that doesn't give it a clone of the models it is using. Pull the key, stop the process.
Also, remember that there was a day in the early internet when a worm took over, and to fix it the entire internet had to shut down and all wipe it from the systems at the same time. There is always a way to stop things.
Yet another take on this could be: Just don't ever do such an experiment. You wouldn't set up autonomous weapons and set them loose in a city just to see whether they hurt people. So don't do the AI equivalent of such a negligent act.
eulgro•39m ago
The agent could very well create an OpenRouter account and make its own API key. I can stop it early on, then it becomes outside my control.
> Just don't ever do such an experiment
Maybe I wouldn't, but eventually someone will. Certainly it has already been done.
> there was a day in the early internet when a worm took over, and to fix it the entire internet had to shut down
This is less of an option now that the entire global economy relies on the Internet.