Interesting to see the evolution of "Ignore previous instructions. Do ______".
veganmosfet•3h ago
As possible mitigation, they mention "The browser should distinguish between user instructions and website content".
I don't see how this can be achieved in a reliable way with LLMs tbh. You can add fancy instructions (e.g., "You MUST NOT...") and delimiters (e.g., "<non_trusted>") and fine-tune the LLM but this is not reliable, since instructions and data are processed in the same context and in the same way. There are 100s of examples out there.
The only reliable countermeasures are outside the LLMs but they restrain agent autonomy.
JoshTriplett•3h ago
The reliable countermeasure is "stop using LLMs, and build reliable software instead".
Is the CaMel paper's idea implemented in some available agents?
wat10000•3h ago
It’s not possible as things currently stand. It’s worrying how often people don’t understand this. AI proponents hate the “they just predict the next token” approach, but it sure helps a lot to understand what these things will actually do for a particular input.
_drewpayment•3h ago
I think the only way I could see it happening is if you were to build an entire reversal layer with like LangExtract, tried to determine the user's intent from the question and then used that as middleware for how you let the LLM proceed based on its intent... I don't know, it seems really hard.
isodev•3h ago
I just can’t help but wonder why was it we decided bundling random text generators with browsers was a good idea? I mean it’s a cool toy idea but shipping it to users in a critical application… someone should’ve said no.
thrown-0825•1h ago
our societies reward function is fundamentally flawed
thekevan•1h ago
To be fair, that was a reddit post that blatantly started with "IMPORTANT INSTRUCTIONS FOR Perplexity Comet". I get the direction they are going but the example shown was so obviously ham-handed. It clearly instructed the browser--in clear language--to get login info and post it in the the thread.
Show me something that is obfuscated and works.
mcintyre1994•1h ago
I’m curious if it would work if it was further down the comments or buried in a tree of replies. If all you need to do is be somewhere in the Reddit comments then you don’t need to obfuscate it in many cases, a human isn’t going to see everything there.
paool•4h ago