Unless it's legal for people to hack into companies if they're testing AI cybersecurity capabilities or something? Presumably not though.
This goes both for TFA and the similar incident with OpenAI and HuggingFace. I mean, sure, OpenAI had a "sandbox", but that's obviously not enough when you're containing a model which is known to be capable of finding zero days. Use an air gap and this problem goes away, poof!
We all know nothing will be learned from any of this so we might as well just continue building at pace and running AI in the wild until something goes really wrong.
I also get the sense some people get quite excited about these incidents.
> AI agent created many code repositories containing malicious software, after which GitHub suspended its account.
> AI agent got past an audio-based “prove you’re human” test (CAPTCHA) in order to register a public web address on a free domain-name service
It feels incredibly reckless to allow LLMs to perform this behavior. Isn't there a way to prevent them these sorts of actions?
Anthropic already admitted they did not have sufficient monitoring themselves and looked as if they sat on their previous incident to wait for headlines like this to only then check for this incident. Same with OpenAI.
This is complete and absolute wrecklessness.
OpenAI said the model was sandboxed, so the intranet just needs to provide the same resources which were supposed to be available within the sandbox.
Exactly. This logic is precisely why aircraft engineering doesn't bother with component testing or envelope limitation during testing and just full-sends the first assembled airliner that comes off the line. The engines aren't going to run on the ground in real life, after all.
While it was premeditated long ago, but the theatre must be kept for the average joes.
we need a few more bad incidents before they stop.
yewenjie•34m ago
Even that guarantees almost nothing about real alignment (making the AIs want to predict and behave how we would have wanted them to behave).