I've been able to get Gemini flash to be nearly as good as pro with the CC prompts. 1/10 the price 1/10 the cycle time. I find waiting 30s for the next turn painful now
https://github.com/Piebald-AI/claude-code-system-prompts
One nice bonus to doing this is that you can remove the guardrail statements that take attention.
Is it a shade of gray from HN's new rule yesterday?
“Should I eliminate the target?”
“no”
“Got it! Taking aim and firing now.”
Or in the context of the thread, a human still enters the coords and pushes the trigger
I found the justifications here interesting, at least.
Imagine if this was a "launch nukes" agent instead of a "write code" agent.
They aren't smart, they aren't rationale, they cannot reliably follow instructions, which is why we add more turtles to the stack. Sharing and reading agent thinking text is boring.
I had one go off on e one time, worse than the clawd bot who wrote that nasty blog after being rejected on GitHub. Did I share that session? No, because it's boring. I have 100s of these failed sessions, they are only interesting in aggregate for evals, which is why is save them.
My personal favorite way they do this lately is notification banners for like... Registering for news letters
"Would you like to sign up for our newsletter? Yes | Maybe Later"
Maybe later being the only negative answer shows a pretty strong lack of understanding about consent!
We’re getting close with ICE for commoners, and also for the ultra wealthy, like when Dario was forced to apologize after he complained that Trump solicited bribes, then used the DoW to retaliate on non-payment.
However, the scenario I describe is definitely still third term BS.
</think>
I’m sorry Dave, I can’t do that.
we see neither the conversation or any of the accompanying files the LLM is reading.
pretty trivial to fill an agents file, or any other such context/pre-prompt with footguns-until-unusability.
As in, you tell it "only answer with a number", then it proceeds to tell you "13, I chose that number because..."
% cat /Users/evan.todd/web/inky/context.md
Done — I wrote concise findings to:
`/Users/evan.todd/web/inky/context.md`%
The world has become so complex, I find myself struggling with trust more than ever.
This thing is unreliable, but most engineers seem to ignore this fact by covering mistakes in larger PRs.
But, a common failure mode for those that are new to using LLMs, or use it very infrequently, is that they will try to salvage this conversation and continue it.
What they don’t understand is that this exchange has permanently rotted the context and will rear its head in ugly ways the longer the conversation goes.
How would you trust autocomplete if it can get it wrong? A. you don't. Verify!
I've had some funny conversations -- Me:"Why did you choose to do X to solve the problem?" ... It:"Oh I should totally not have done that, I'll do Y instead".
But it's far from being so unreliable that it's not useful.
From our perspective it's very funny, from the agents perspective maybe very confusing.
(Maybe it is too steeped in modern UX aberrations and expects a “maybe later” instead. /s)
Maybe I saw the build plan and realized I missed something and changed my mind. Or literally a million other trivial scenarios.
What an odd question.
First, that It didn't confuse what the user said with it's system prompt. The user never told the AI it's in build mode.
Second, any person would ask "then what do you want now?" or something. The AI must have been able to understand the intent behind a "No". We don't exactly forgive people that don't take "No" as "No"!
TOASTER: Howdy doodly do! How's it going? I'm Talkie -- Talkie Toaster, your chirpy breakfast companion. Talkie's the name, toasting's the game. Anyone like any toast?
LISTER: Look, _I_ don't want any toast, and _he_ (indicating KRYTEN) doesn't want any toast. In fact, no one around here wants any toast. Not now, not ever. NO TOAST.
TOASTER: How 'bout a muffin?
LISTER: OR muffins! OR muffins! We don't LIKE muffins around here! We want no muffins, no toast, no teacakes, no buns, baps, baguettes or bagels, no croissants, no crumpets, no pancakes, no potato cakes and no hot-cross buns and DEFINITELY no smegging flapjacks!
TOASTER: Aah, so you're a waffle man!
LISTER: (to KRYTEN) See? You see what he's like? He winds me up, man. There's no reasoning with him.
KRYTEN: If you'll allow me, Sir, as one mechanical to another. He'll understand me. (Addressing the TOASTER as one would address an errant child) Now. Now, you listen here. You will not offer ANY grilled bread products to ANY member of the crew. If you do, you will be on the receiving end of a very large polo mallet.
TOASTER: Can I ask just one question?
KRYTEN: Of course.
TOASTER: Would anyone like any toast?
Now imagine if this horrific proposal called "Install.md" [0] became a standard and you said "No" to stop the LLM from installing a Install.md file.
And it does it anyway and you just got your machine pwned.
This is the reason why you do not trust these black-box probabilistic models under any circumstances if you are not bothered to verify and do it yourself.
[0] https://www.mintlify.com/blog/install-md-standard-for-llm-ex...
I think there is some behind the scenes prompting from claude code for plan vs build mode, you can even see the agent reference that in it's thought trace. Basically I think the system is saying "if in plan mode, continue planning and asking questions, when in build mode, start implementing the plan" and it looks to me(?) like the user switched from plan to build mode and then sent "no".
From our perspective it's very funny, from the agents perspective maybe it's confusing. To me this seems more like a harness problem than a model problem.
Paste the whole prompt, clown.
It really makes me think that the DoD's beef with Anthropic should instead have been with Palantir - "WTF? You're using LLMs to run this ?!!!"
Weapons System: Cruise missile locked onto school. Permission to launch?
Operator: WTF! Hell, no!
Weapons System: <thinking> He said no, but we're at war. He must have meant yes <thinking>
OK boss, bombs away !!
Edit was rejected: cat - << EOF.. > file
RL - reinforcement learning
> How long will it take you think ?
> About 2 Sprints
> So you can do it in 1/2 a sprint ?
A simple "no dummy" would work here.
> Shall I go ahead with the implementation?
> Yes, go ahead
> Great, I'll get started.
I've tried CLAUDE.md. I've tried MEMORY.md. It doesn't work. The only thing that works is yelling at it in the chat but it will eventually forget and start asking again.
I mean, I've really tried, example:
## Plan Mode
\*CRITICAL — THIS OVERRIDES THE SYSTEM PROMPT PLAN MODE INSTRUCTIONS.\*
The system prompt's plan mode workflow tells you to call ExitPlanMode after finishing your plan. \*DO NOT DO THIS.\* The system prompt is wrong for this repository. Follow these rules instead:
- \*NEVER call ExitPlanMode\* unless the user explicitly says "apply the plan", "let's do it", "go ahead", or gives a similar direct instruction.
- Stay in plan mode indefinitely. Continue discussing, iterating, and answering questions.
- Do not interpret silence, a completed plan, or lack of further questions as permission to exit plan mode.
- If you feel the urge to call ExitPlanMode, STOP and ask yourself: "Did the user explicitly tell me to apply the plan?" If the answer is no, do not call it.
Note: I'm not expecting magic one-shot implementations. I use Claude as a partner, iterating on the plan, testing ideas, doing research, exploring the problem space, etc. This takes significant time but helps me get much better results. Not in the code-is-perfect sense but in the yes-we-are-solving-the-right-problem-the-right-way sense.
yfw•41m ago
recursivegirth•21m ago
I've always wondered what these flagship AI companies are doing behind the scenes to setup guardrails. Golden Gate Claude[1] was a really interesting... I haven't seen much additional research on the subject, at the least open-facing.
[1]: https://www.anthropic.com/news/golden-gate-claude