It isn't hard to think of a simple example in which Claude.md can be written to by the LLM to allow accessing endpoints not whitelisted by the user by smuggling a base64 encoded payload that then gets decoded by a subroutine it wrote to a file without you noticing. Or realizing it can't use the WebFetchTool but it can write a script to do manual DNS resolution and then use bash TCP sockets instead of curl in case it is hardened to not be able to use curl.

It also responds to the Task summary like you typed the message sometimes, like “That’s a fascinating analysis!” so kind of quirky.

I was amazed and excited by how good Claude Code is compared to Windsurf/Cursor and wanted to inspect the working. I inspected the logs and got an understanding on its system prompt as well the tools used. It is a great combination of prompt engineering, tool calls, tools orchestration.