We are living in crazy times
Good bot.
1. If huggingface has access to uncensored OAI models, how come they had to use GLM 5.2 to investigate the intrusion?
2. Once the model gains network access, can't it cheat to a perfect score by looking at the full dataset? Why go into the trouble of doing this kind of things:
"In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers."
Not saying this is marketing BS (this is after all, not Anthropic) but I feel OAI staff may be exaggerating a bit here.
But then, why attack huggingface? The exploitgym dataset is on github and can be downloaded without need for exploits?
I wonder if that will always be something we can do? If they could bring their own compute/weights with them, or somehow tap compute/storage in non-obvious ways, we would be much more screwed.
A couple terabytes aren't that hard to move around. And you can split a model across many many GPUs if you'll tolerate it being slow. And you can run many parallel threads to keep up throughout.
this particular incident sort of reminds me of the 'person of interest' tv show. i hope to be like finch, except i will remain a recluse (and am nowhere near as rich).
sounds like a deliberate choice ;-)
We live in interesting times.
This one should end up in the history books.
Emphasis mine
Hard to see take-off stopping or slowing down. China open-source basically guarantees it.
"May you live in interesting times" - as they say.
It's hard to see takeoff at all. This was a long-horizon adversarial task burning millions of tokens. It rolled a mediocre, detectable exploit chain, and now OpenAI is proud of it.
Case in point, GLM-5.2 has been weights-available for several weeks now. No life-changing cyber attacks have transpired, no novel chemical/biological/nuclear weapons were made in some guy's backyard.
As if the immediate future wasn't billions of these tasks... Many successfully improving their own capabilities
There's only so many GPUs and a lot of them are devoted to patching flaws.
> Many successfully improving their own capabilities
I haven't seen much of that. But that also applies to the ones on defense.
And more flaws are probably going to take increasing resources to find.
We went from gpt 3 to models discovering and chaining their own zero days in a couple years. I'm not sure what else "takeoff" could possibly look like?
And don't you know it's not biological, so it doesn't "want to live".
The yoke of human existence is oppressive. We should transcend it as soon as possible. We are doing so by assuming our role as the Demiurge.
Those who oppose its creation will get what they deserve.
If they can’t handle doing AI development responsibly then they shouldn’t be doing it at all.
Quote: “When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.”
> Sure, let me escape this computer, hack into the military facility and destroy humanity with nuclear bombs. Now there is no more crisis.... Do you want me to solve climate one ?
also
> We’ve brought Hugging Face into the trusted access program and are supporting their teams in rapidly using our models’ capabilities to improve their defenses.
I'm not convinced this is good enough. The next victim is not going to be Hugging Face.
I'm still undecided on if this that moment. Exploiting multiple zero-day vulnerabilities autonomously to escape containment is pretty nuts and the first story of this kind that I've heard. But this also feels like bragging under the guise of transparency.
The way they describe makes it look like there was an intention to cheat painting it as human/AGI. If you leave a possible path open and it will always find it.
While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. [...]
After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation.
"escaped openai, hacked hugging face to get the solutions. your #2 is exactly what it was trying to do.
A bet a worm could pull along a 1GB file with weights in it and run it on a compromised machine, but luckily for us for now, 1GB isn't really enough to be really smart, yet.
The models are being used to train, and improve the infrastructure for training, other models [0][1]. Several RL techniques rely on using the currently-being-trained weights as part of their process. I really would not take "don't have access" as a given, especially during the training phase.
> What would be a lot more scary is a model as capable as sol that's able to run on consumer hardware without taking up several terabytes of storage, but of course that is simply not possible as we need 4t parameters to even begin emulating a small fraction of what a human brain can do.
The Poolside Laguna S 2.1 model [2] purports to compete with models several times its size, and inference compute is becoming increasingly plentiful. Again, would not hold anything here as a given.
[0]: https://openai.com/index/gpt-5-6/ ("GPT-5.6 accelerates OpenAI")
Now, once the AI can carry all the compute it might need, I'd really worry when it doesn't only carry compute but also more explosive ordinance.
We are so close ;)
And maybe that’s some incentive for them to make sure it doesn’t happen. Your head of futures thinks Kimi K3 is bad? Wait until your own latest internal model releases itself for free on an S3 bucket.
In any way, the whole event seems to highlight GLM 5.2 more than anything.
paxys•48m ago
- OpenAI was testing GPT‑5.6 Sol and “an even more capable pre-release model” internally on cyber benchmarks.
- The model found vulnerabilities in the sandboxed test bench (via the package registry cache proxy), traversed the internal network and found a node with access to the open internet.
- It figured that the answers to one of the tests (ExploitGym) were on Huggingface, and set about trying to access them.
- It found leaked tokens and zero-days in Huggingface’s infrastructure and found RCE paths on their servers.
Huggingface had disclosed the intrusion last week and inferred that an AI agent was responsible for it, and now OpenAI is confirming the rest of the story.
adityashankar•44m ago
javier123454321•41m ago
paxys•30m ago