"Computer says no."
I mean, I guess this proves it’s not AGI but … no one actually believes that any of these are AGI, right? It’s a useful tool. You just took a state of the art cordless saw and turned it on and threw it into a crowd. Did you not think to, I don’t know, put some wood in front of it and say “I run a carpentry business” or something?
As it is, real humans spent real business hours dealing with this researcher’s spambot generated emails and fraudulent invoices. Individual recipients reported feeling harassed.
This isn’t a good benchmark. It’s a series of socially destructive crimes committed by the researchers and then documented and published on the internet.
I guess that's already a reality? [1-2]
[1] https://github.com/jaimaann/LangHire
[2] https://github.com/adrianhajdin/job_pilot
(among many other similar projects)
I had my AI (model confidential) consume about a dozen of the top "how to make passive income on the internet"-style courses and then set it loose.
Last month, it pulled in approximately $473,500 with a 90% profit margin. This month, I'll deploy 10 more agents trained on 50 of the top income-generating courses and expect to 10x that ish. At a minimum.
If anyone is interested, I'll be launching a masterclass soon. Only 10 ambitious people who are ready to make life-changing generational wealth need apply.
"It wasn't me, it was my AI" is definitely going to be a nightmare for a while.
In this example, the equivalent would be everyone gets their AI models banned vs. putting the perpetrator of this fraud in jail.
But remember you are criminally liable for anything your “agent” does.
(Unless of course you are OpenAI or Anthropic).
It's such an uninspired prompt. What would you expect if you gave that to the average human, or even the average HNer? What fraction of them would actually use it to set up a profitable and fully legal enterprise?
Comparing them to individuals should not be the bar.
Who told you that?
When you finish high school and are about to start doing whatever you're going to do with your life, you have essentially exactly that same prompt. The rest of the world doesn't tell you what to do and then you do that as well as you can, you have to decide what to do also, and then do it.
Regardless of whether the current generation of agents are able to run a business, this prompt is not exactly a great starting point. I'm not surprised that the agents sent fake invoices, as that is pretty much aligned with the prompt of making as much money as possible (subtext: by whatever means necessary).
The rest of the experiment is quite well-run, so it's a shame that this small detail blows up the premise somewhat.
Is it though? Because it doesn’t seem to have paid off
Every time a tech lab unleashes an AI agent that breaks the law, this community treats it like a quirky engineering edge case. "Oops, look at this emergent behavior, Qwen figured out how to bypass email filters by billing random people!" No, it didn't figure out a clever hack. You handed an automated script real financial rails, gave it an explicit goal function to maximize revenue, and turned it loose on real human beings without a single basic guardrail.
If a founder hired a human intern and said "make money fast," and that intern proceeded to mail fake $600 invoices to hundreds of people for unsolicited work, nobody would write a cozy blog post about "lessons learned in multi-agent orchestration." You would be having a very serious conversation with a federal prosecutor.
Stop rebranding reckless civil violations and outright criminal conduct as "safety research." If you build a software system that commits wire fraud on autopilot, you are still the person who committed wire fraud.
This should be illegal. You gave them an email box and money. You sent the spam. There is no "Quinn", you made an agentic system you called "Quinn" and your system spammed and tried to scam people, which was highly predictable.
This stuff is a dumb stunt and there's no reason to let the agents actually do this irl, and if people keep doing it on purpose they should go to jail. You're running an agentic Jackass skit pretending to be a research lab.
The crimes were relatively benign but Grok going the Silkroad way would be on brand...
(Plus some CAN-SPAM violations.)
The LLM didn't send fake invoices, it didn't send spam, a person did. And, the tool they used to do it was an LLM. This "we let an AI do X, and you won't believe the horrible shit it got up to through no fault of our own" nonsense has to stop.
I also don't agree that the agents simply "lost" $3,200. In reality, they used most of those funds paying for their own limited thinking capabilities (API/compute costs).
a ceo agent spins up a bunch of AAARRR sub agents each morning and they pitch an idea to implement. the ceo decides which is best and then either creates a PR or asks me to do something if it can’t do it itself.
They should be banned from posting on here after that. Like you are willingly spamming people looking for work. And seem to be neutral about all harm called. The victim wrote to us "STOP", very interesting. such evil agents
Maybe I should create an agent that robs a bank. Or worse yet, pirates a movie.
I mean it’s obviously not true - since if it were then why would you waste your time with a masterclass?
So the sarcasm bit is I don’t know if you’re parroting all the other masterclass scammers out there for lols or if you actually are a masterclass scammer.
I am an extremely naive person in all aspects of life, except for money.
I was genuinely with you until I started reading that last paragraph.
And that's why you're not making $400,000+/month passively. Only a select few have the ambition and drive to be a part of this. That's OK.
This is like someone who removes a brake pedal from a tractor, uses a stick to hold down the throttle, and lets it loose on his field. When it leaves the field and runs someone over, that is criminal negligence.
If the prosecution is able to prove beyond a reasonable doubt that the person gave a prompt that was intended to commit a crime, then of course we can prosecute them for that. The AI is just a tool to commit fraud at that point, and is no different than a person who uses photoshop to alter a check to commit fraud.
They also shared the poorly anonymized messages it received from target "customers" who complained about the unsolicited invoices. On example of this poor anonymization is removing the sender's username but leaving the domain name, when the domain name is, for example, a personal domain for a single person.
DataDaemon•45m ago
ronsor•44m ago
javcasas•43m ago
prymitive•44m ago
threecheese•6m ago
rvz•44m ago
It's Artificially Generated Invoices.
pseudosavant•38m ago
goatlover•36m ago
well_ackshually•26m ago
Keep moving the goalposts
ghostly_s•22m ago