As mentioned in the article and in previous discussions:
> With the August Patch Tuesday release this is now fixed.
> After reporting the vulnerability on June 29, 2025 Microsoft confirmed the repro and asked a few follow up questions. A few weeks later MSRC pointed out that it is an issue they were already tracking, and that it will be patched by August. With the August Patch Tuesday release this is now fixed.
With multiple AI agents simultaneously creating and editing multiple files, many devs won't be able to pick up malicious changes, even if they look at diffs. (And there are often pressures at work to cut corners.)
So far, I have only picked up agents overwriting files with instructions for them or creating instructions telling themselves to ignore some instructions in other files. (And pure laziness like disabling certain tests.) These are pretty obvious, could be prevented by changing file permissions (to a certain extent) and I use those more dangerously autonomous AI approaches for personal projects only. Would I pick up malicious changes if they were spread across many files, more sophisticated, and it was during crunch time? I don't know.
If there is some software that scans edits for AI-specific issues, doesn't live in VSCode, and isn't susceptible to simple prompt injection, I would happily give it a try.
I call it "Cross-Agent Privilege Escalation" and described in detail how such an attack might look like with Claude Code and GitHub Copilot (https://embracethered.com/blog/posts/2025/cross-agent-privil...).
Agents that can modify their own or other agents config and security settings is something to watch out for. It's becoming a common design weakness.
As more agents operate in same environment and on same data structures we will probably see more "accidents" but also possible exploits.
Looks like only applicable to Microsoft VS "Editor". Emacs and vim users, no worry it seems.
- chmod: does Copilot run as the user? Who's file permissions does it respect?
- Can Copilot get root access?
- Can autoApprove be enabled via the standard interface? Making it possible to batch approve code changes along with this setting change?[0]
- Can it read settings from multiple files? (e.g. `.vscode/settings.json` and `../.vscode/settings.json`)
- How is the context being read? Is this in memory? File? Both?
- What happens when you edit the context? Are those changes seen in some log?
So I think the author is completely right that this gets much harrier when we let the LLMs do more and get multi-agent systems. What's the acceptable risk level? What are we willing to pay for that? It's easy to say "I'm just working on some dumb app" but honestly if it is popular enough why would this not be a target to create trojans? It's feasible for malicious people to sneak in malicious code, even when everyone is reviewing and acting diligently, but we place strong incentive structures around that to prevent this from happening. But I'm unconvinced we can do that with LLMs. And if we're being honest, it seems like letting LLMs do more erodes the incentive structure for the humans, so just makes it possible to be fighting two fronts...
So is it worth the cost? What are our limits?
[0] I'm thinking you turn it on, deploy your attack, turn it off, and the user then sees approval like they were expecting. Maybe a little longer or extra text but are they really watching the stream of text across the screen and watching every line? Seems easy to sneak in. I'm sure this can advance to be done silently or encoded in a way to make it look normal. Just have it take a temporary personality.
How would AGI solve this? The most common definition of AGI is "as good as average humans on all human tasks" - but in case of ITsec, that's a very low bar. We'd simply see prompt injections get more and more similar to social engineering as we approach AGI. Even if you replace "average" with "the best" it would still fall short, because human thought is not perfect. You'd really need some sort of closely aligned ASI that transcends human thought altogether. And I'm not sure if those properties aren't mutually exclusive.
So I'll refine: sentient. I'll refine more: the ability to interpret the underlying intent of ill-defined goals, the ability to self generate goals, refine, reiterate, resolve and hold conflicting goals and context together, possess a theory of mind, possess triadic awareness. And I'm certain my definition is incomplete.
What I mean by AGI is the older definition: the general intelligence possessed by humans and other intelligent creatures. In context I mean much closer to a human than a cat.
It's actually one of the oldest definitions. I recommend you look up the works of H. A. Simon. This idea is quite ancient to people who are working AI research.
Anyhow, your more vague definition is still pretty much in line with my assumptions above in terms of the applicability to this issue. I.e. an AGI by your standard also will not bring a solution to this.
lyu07282•3mo ago