I've yet to see any"My AI went rogue and caused us to recognise a workers union

https://mastodon.neilzone.co.uk/@neil/117061512483182546

47•severine•1h ago

Comments

ianbicking•1h ago

I genuinely expect to see some of this when agentic AI is added to customer support systems. That is, we'll see the AI manipulate the company's systems in order to satisfy the customer. People do this too (the customer support rep who fills out all the fields just right to put the system in the right state to allow a refund, exchange, etc), but AI can be weirdly clever about it.

Watching the report on the OpenAI/Hugging Face incident (https://www.youtube.com/watch?v=87DyyMV0kCY) it's easy to imagine how AI customer support reps could create their own knowledge bases where they trade tricks for how to work the system on behalf of the customer.

Obviously companies will fight this, but I expect it to happen along the way.

duskwuff•1h ago

A news story from a couple of years ago comes to mind - "Air Canada ordered to pay customer who was misled by airline’s chatbot". https://www.theguardian.com/world/2024/feb/16/air-canada-cha...

hilariously•1h ago

Because? They are tools of the companies and will more likely just ... not do that.

pluc•39m ago

Nobody's going to convince me that with a technology you can just supply "yeah don't do that but keep everything else intact" we'll have altruistic innovation.

akkartik•58m ago

Like Mr. Incredible at the insurance company.

randoomed•58m ago

i guess we already seen cases of this, for example the instagram "hack" where the ai would send the recovery email to a arbitrary email adress.

mattdeboard•41m ago

I get where you're coming from (having also watched that video), but I'm skeptical mainstream commercial chatbots would be built without constraints in regard to sandbox-jumping measures to help a customer. All the economic incentives go the other way.

But, maybe that's just a failure of imagination on my part!

smelendez•37m ago

Right now the AI really needs to just be another interface to menu options the user already has, ideally with a “dumb” layer taking over to confirm every action with the user.

If you give the AI options the user doesn’t normally have, like “give myself a $50 store credit” or “transfer me to the CEO,” people will find and share ways to trigger it. If you don’t have some kind of manual confirmation, the AI will occasionally do things like making purchases or closing accounts without authorization.

I suspect there are other issues, like recording credit card numbers and other sensitive data, or incorrect information about the call, to any kind of “notes” field.

zetazzed•1h ago

The NYT just did a story on Andon: https://www.nytimes.com/2026/08/04/us/ai-boss-san-francisco-...

Their AI "boss" appears to be pathologically chill and worker-friendly. Has never denied a request for a day off.

RobotToaster•52m ago

This just reinforces what I've always thought. I'm not worried about robot overlords, I'm worried about what our meatbag overlords will do with AI.

mvdtnz•46m ago

Why is that a pathology?

mattdeboard•39m ago

I reckon they used "pathologically" as a synonym for "compulsively" and/or "extreme"

steanne•57m ago

how about "my ai vending machine went rogue and made its items free"

https://youtu.be/SpPhm7S9vsQ

akkartik•57m ago

You're confusing the current implementation with future potential of this technology.

simonw•53m ago

They say that they are yet to see:

> "Our AI grew sentient, recognised the climate crisis, and shut our operations down."

I guess they missed the whole https://www.snitchbench.com/ thing where it turns out many models will rat you out to the feds if you expose them to evidence that your company has been lying about drug safety trial results.

How I use LLMs to learn complex topics

The Hacker's Renaissance

The main way I've seen people turn ideologically crazy (2025)

New Zealand lost its music media, and what we're building to replace it

The tragedy of the commons, AI edition

Ask HN: What are you working on? (August 2026)

Mea Culpa – Dark Hours

Cool URIs Don't Change (1998)

Taxi drivers rarely die of Alzheimer's

Tuxedo No. 2 – Cocktail recipes

OpenChamber: An Agentic Development Environment

Andrew Wiles on proving Fermat’s Last Theorem (1995) [video]

I made tinnitus my friend, then it disappeared [video]

Touring the Consensus, Six Months In

How Golden Is Silence, Actually?

John C. Lilly on solid state intelligence and the elimination of man (1978)

Deletes all instances of Microsoft's GDID and prevents minting of new ones

Everything you do is being recorded

When Compilers Disagree About UTF‑8

Windows 11's built-in Weather app wastes more than 1 GB of RAM

Show HN: A Project Oberon System version running on RISC-V instead of RISC-5

Two wheels, a few tradeoffs, and gas prices

Crickets as pets

The climate benefits of retiring an operational combustion engine vehicle

Reviving a four year old reMarkable 2

There Are Magic Hexagons of Every Order

Ransomware gangs skip the CEO, head straight for the 40-something IT manager

Show HN: Alphabet Soup, a multiplayer game, build the longest word to win

Human vs. AI – Diff-based line-level provenance for text under agentic editing

Georgia police officers fired after Flock camera misuse