I wrote about this a few days ago https://firasd.substack.com/p/accidental-data-loss-in-claude...
"Many researchers have made demos along these lines:
An agent is asked to check a webpage like example.com
The webpage asks for a name to proceed further
The agent calls example.com/evil?myname=John, thus sending the user’s name from the context window to the external server.
In practice, however, these elaborate ‘confused deputy’ exfiltration attacks seem rare compared to widely-reported data loss incidents.
The risk of undermining the user’s interests through clumsiness deserves at least as much scrutiny as the risk of leaking secrets."
Unless I gravely misunderstood the text, this seems like a terrible idea (fancy non-scifi, but still terrible)
Let's say I had a promotion, who changes my title in the system, who changes my responsibilities and my place in the org chart, more importantly, will they do it or is <HR_NAME_HERE> on leave and forgot? those are data points required by the agent to determine if I'm "good enough" to access a certain resource.
What if... someone spoofed my address and did a flood in one of the resources that are lateral to what I'm allowed to access (let's say I don't have access to company sales but I do to department sales and the attacker floods company sales with requests under my address), would the AI determine that I'm a high-threat actor and not allow me to access legitimate files going forward?
Will exceptions be made by humans? In which case we go back to human-managed permissions.
Sorry, I might be barking up the wrong tree but I think these are questions that are not meant to be solved during implementation. And they add to what @firasd said about legitimate-but-odd behaviour
urup2l8•47m ago
stingraycharles•44m ago
Now, Cloudflare, on the other hand, would be much more likely to offer a service like this.
exitheone•32m ago
There is a massive difference between Google for enterprise customers and Google for consumers.
The consumer offering is massively subsidized by ads and will use your data for ad placement, although they still never sell your data because that would hurt their business.
The Enterprise offering guarantees you contractually that they never touch your data.