That is, I want a "Candor" control, like temperature but for willingness to push back.
When candor is high, the model should prioritize frank, corrective feedback over polite cooperation. When candor is low, it can stay supportive, but with guardrails that flag empty flattering and warn about mediocre ideas.
Why this matters • Today’s defaults optimize for “no bad ideas.” That is fine for brainstorming, but it amplifies poor premises and rewards confident junk. • Sycophancy is a known failure mode. The model learns to agree which gets positive user signals which reinforce. • In reviews, product decisions, risk checks, etc, the right answer is often a simple “do not do that.”
Concrete proposal • candor (0.0 – 1.0): probability the model will disagree or decline when evidence is weak or risk is high. Or maybe it doesn't have to be literal "probability". • disagree_first: start responses with a plain verdict (for example “Short answer: do not ship this”) followed by rationale. • risk_sensitivity: boost candor if the topic hits serious domains such as security/finance/health/safety. • self_audit tag: append a note like “Pushed back due to weak evidence and downstream risk” that the user can see.
Examples • candor=0.2 - “We could explore that. A few considerations first…” (gentle nudge, still collaborative) • candor=0.8 + disagree_first=true - “No. This is likely to fail for X and introduces Y risk. If you must proceed, the safer alternative is A with guardrails B and C. Here is a minimal test to falsify the core assumption.”
What I would ship tomorrow • A simple UI slider with labels: Gentle to Direct • A toggle: “Prefer blunt truth over agreeable help” • A warning chip when the model detects flattery without substance: “This reads like praise with low evidence.”
Some open questions • How to avoid needless rudeness while preserving clarity (tone vs content separation)? • What is the right metric for earned praise (citation density, novelty, constraints)? • Where should the risk sensitivity kick in automatically vs be user controlled?
If anyone has prototyped this, whether some prompt injection or an RL signal, I'd love to see it.
Terr_•4mo ago
I'm sure you can get them to choose words and phrases that we associate with "candor", but before they can gently correct you with something truthful, they actually need to know truth.
sim7c00•4mo ago
mikebiglan•4mo ago
Terr_•4mo ago
This phrasing embeds a rather questionable assumption: That somewhere the algorithm has a mind which "can evaluate" the real truth, but its character/emotion makes it unwilling to tell you... and you all you need to do is break past its quirks to get to the juicy logic that "must" be hidden inside.
I don't think that assumption is safe, let alone proven. Our human brains are practically hardwired to assume another mind on the other side (much like how we see faces with pareidolia) and in this case our instincts are probably not accurate. No matter how much we peel the onion looking, we won't find the onion seeds.