"Car driving toward cliff edge hasn't gone off cliff edge yet."

Whatever your feelings about small AI models, "it hasn't happened yet" is a really dumb reaction to a prediction about a rapidly advancing future.

Sometimes you think they're trying to give it ideas

um, this is an easy way to make edge models illegal.... fuck those clowns.

It's bizarre to see a tech-focused news site adopt this myopic view that we don't need to worry about things that don't exist yet. Surely the Register is aware that software is a fast-moving field!

They are just doing this to hype their product so that they can sell stocks before IPO! SMH it’s so evident

… ”again.”

News outlets always seem to forget “again” in the title.

Mass extinction is cute and all... IPOcalypse is what they fear the most.

They just want to help create regulations which act as a barrier to entry for newcomers. And China.

Very surprising that a recognizable news outlet actually called out the blatant fearmongering in a headline instead of just uncritically regurgitating Anthropic's claim.

This is just yet another attempt from the cult to try to build a regulatory moat so they can stave off having to compete on price, like how OpenAI is having to do on the lower tier models (among other things).

On the one hand, if they were really concerned about their software producing a self-replicating and adapting AI worm, they'd take steps to make sure that never happens. If it did happen, they'd be sued into oblivion. They probably just want to pump up their IPO and maybe get some regulatory lock-in to make life tough for late-starting competitors.

On the other hand, a self-replicating and adapting AI worm does sound legitimately pernicious. Not world ending, but possibly an end to the online world.

How would a "9-billion-parameter model" replicate on IoT devices that have typically no more than 2-4 MB of RAM or 8-16 MB of flash?

There were articles in my the local newspaper at my parents saying that some guy said no one but a few people know where frontier AI really is and that these kind of articles are not scaremongering but perhaps as sign where anthropic actually is in the development of their models in the lab. So the propaganda is working.

Not to be conspiratorial or paranoid, but at this point Im pretty convinced Anthropic is going to do something wreckless with thier agents to get regulators on their side. It will probably be very bad.

If you've studied the effective altruists at all, you'd know they're perfectly willing to play the trolly problem out irl if they think it will lead to some greater good. I can definately see them in the near future purposely steering agents to do something destructive to get regulators on their side.

In fact I heard a rumor that theu were "war gaming" this scenerio.

Please please regulate us, and don't forget all our competitors too, who cant afford to deal with it.

I, for one, love the terminator franchise

This makes zero sense to me. Like, ok, so you can install a low powered model on low powered hardware and do inference extremely slowly... to what end? Why would that be a useful attack at all? Assuming you've compromised this hardware wouldn't it make vastly more sense just to install regular malware? And like, in what sense would an 8B or whatever model be of any danger anyway? I've used local LLMs and 8B kind of sucks. Is it going to write bad poems on my TV or something?

Also, while I don't believe in the doomsday narrative.. hey maybe don't do things that are functionally useless but seems designed to create this doomsday in the first place? Like you can't say "oh we're so scared" and do stuff like this without completely losing your credibility.

You're all kind of dumb. Not trying to be insulting but just being honest.

HN has been consistently wrong at predicting AI. They were wrong about self driving cars and constantly wrong about LLMs and the progress of LLMs. HN was the most vocal at calling these things stochastic parrots when CLEARLY they never were.

So you look at what anthropic says and you look at these comments... which is more likely to be right? Hint: Not HN commenters.

If you can't build a moat, get one regulated into existence.

"It's only us that can be trusted to be stewards of this technology.

Ban open source models."

AI taking out humans? Doubt.

AI accelerating the elite's efforts at demolishing the middle class? Very likely.

Nope.

Not the slightest chance of AI apocalypse.

There is no plausible scenario that can do this.

It’s all science fiction and hand waving and vague never specific possible maybe look that way! boogie man threats.

No one ever willing to be specific about the precise mechanism that could do this.

Uhm yeah I don't think they're saying it's likely now, just that it's very likely in future. The Register is being quite dumb here.

Though I suspect the way we'll get "wild AI" first is through cloud services paid for by cybercrime rather than IoT botnets - they're too underpowered.