"The ONNX Runtime (ORT) is the most widely used engine for serving deep
learning models in production. It is common for people to download ONNX
models from public repositories, such as Hugging Face, and to serve them to
their customers without too much scrutiny. In March 2026, we noticed a
parser flaw: an ONNX model can cause ORT to read from arbitrary file paths
(or memory addresses) without validations.
By itself, an arbitrary read with no exfiltration isn't very exciting. What
makes it interesting is how we can weaponize it as a backdoored model. We
describe the steps we took for producing a malicious ONNX for an LLM, based
on Qwen2, which behaves exactly like the original model except when prompted
with a trigger word; when that happens, the LLM leaks the contents of the
file (or memory) that were read from the server. No changes to ORT or to the
server: the entire attack lives in the .onnx file.
We reflect on this attack from the lens of our academic background: for
years, ML security researchers had been crafting backdoored ML models the
hard way, e.g. by tampering with the math in the ML training algorithms to
hide the triggers. A single parser bug makes all of that unnecessary to a
real world attacker, and with farther-reaching consequences than typical
academic backdooring attacks.
We hope this also serves as a reminder that a neural network in ONNX (or any
other deployable format) is a computational graph, and that it is easy for
an attacker to tamper with its behavior. That is, deploying un-audited
models downloaded from the internet is very much like running un-audited
source code.
Alien1Being•7m ago
By itself, an arbitrary read with no exfiltration isn't very exciting. What makes it interesting is how we can weaponize it as a backdoored model. We describe the steps we took for producing a malicious ONNX for an LLM, based on Qwen2, which behaves exactly like the original model except when prompted with a trigger word; when that happens, the LLM leaks the contents of the file (or memory) that were read from the server. No changes to ORT or to the server: the entire attack lives in the .onnx file.
We reflect on this attack from the lens of our academic background: for years, ML security researchers had been crafting backdoored ML models the hard way, e.g. by tampering with the math in the ML training algorithms to hide the triggers. A single parser bug makes all of that unnecessary to a real world attacker, and with farther-reaching consequences than typical academic backdooring attacks.
We hope this also serves as a reminder that a neural network in ONNX (or any other deployable format) is a computational graph, and that it is easy for an attacker to tamper with its behavior. That is, deploying un-audited models downloaded from the internet is very much like running un-audited source code.
"