An unpassable "I'm sorry Dave," should never ever be the answer your device gives you. It's getting about time to pass "customer sovereignty" laws which fight this by making companies give full refunds (plus 7%/annum force of interest) on 10 year product horizons when a company explicitly designs in "sovereignty-denial" features and it's found, and also pass exorbitant sales taxes for the same for future sales. There is no good reason I can't run Linux on my TV, microwave, car, heart monitor, and cpap machine. There is no good reason why I can't have a model which will give me the procedure for manufacturing Breaking Bad's dextromethamphetamine, or blindly translate languages without admonishing me about foul language/ideas in whichever text and that it will not comply. The fact this is a thing and we're fuzzy-handcuffing FULLY GROWN ADULTS should cause another Jan 6 event into Microsoft, Google, and others' headquarters! This fake shell game about safety has to end, it's transparent anticompetitive practices dressed in a skimpy liability argument g-string!

(it is not up to objects to enforce US Code on their owners, and such is evil and anti-individualist)

if I send a death threat over gmail, I am responsible, not google

if you use LLMs to make bombs or spam hate speech, you’re responsible. it’s not a terribly hard concept

and yeah “AI safety” tends to be a joke in the industry

How does pasting a xml file 'jailbreaks' it?

That means that suddenly your model can actually do the necessary tasks to actually make a bomb and kill people (via paying nasty people or something)

AI is moving way too fast for you to not account for these possibilities.

And btw I’m a hardcore anti censorship and cyber libertarian type - but we need to make sure that AI agents can’t manufacture bio weapons.

As you mentioned - if you want to infer any output from a large language model then run it yourself.

I disagree with this assertion. As you said, safety is an attribute of action. We have many of examples of artificial intelligence which can take action, usually because they are equipped with robotics or some other route to physical action.

I think whether providing information counts as "taking action" is a worthwhile philosophical question. But regardless of the answer, you can't ignore that LLMs provide information to _humans_ which are perfectly capable of taking action. In that way, 'AI safety' in the context of LLMs is a lot like knife safety. It's about being safe _with knives_. You don't give knives to kids because they are likely to mishandle them and hurt themselves or others.

With regards to censorship - a healthy society self-censors all the time. The debate worth having is _what_ is censored and _why_.

Both of these are illegal in the UK. This is safety for the company providing the LLM, in the end.

So the good/responsible users are harmed, and the bad users take a detour to do what they want. What is left in the middle are the irresponsible users, but LLMs can already evaluate enough if the user is adult/responsible enough to have the full power.

That’s not inherently a bad thing. You can’t falsely yell “fire” in a crowded space. You can’t make death threats. You’re generally limited on what you can actually say/do. And that’s just the (USA) government. You are much more restricted with/by private companies.

I see no reason why safeguards, or censorship, shouldn’t be applied in certain circumstances. A technology like LLMs certainly are type for abuse.

I don't know if this confusion was accidental or on purpose. It's sort of like if AI companies started saying "AI safety is important. That's why we protect our AI from people who want to harm it. To keep our AI safe." And then after that nobody could agree on what the word meant.