This is an issue for tasks like content moderation and labelling. Judgements like this are subjective, highly dependent on context and generally messy.
Theoretically, you supply a policy and content, and the LLM labels according to the policy. In practice, the model has inertia which means you don’t get what you expect. Your large 5 page policy document only provides a minor improvement over a one line policy.
The other issue is that you may create carve outs for content in your policy, but the model will still flag it as violative. No matter how strong the carve out.
The most recent work I know of here is Zentropi’s policy steerability benchmark. They give a model the same content under two policies — one that says flag, one that says allow — and only score the pairs where it gets both right
If I am reading the numbers correctly, Opus-4.6 lands at 0.52 steerability — but that’s 0.97 positive accuracy against 0.54 negative. It flags almost everything it should, but 47% of the time when it shouldn’t. Sonnet, which is more deferent, is (somehow) less steerable.
I think this also implies that safety and Steerability are antagonistic to each other.
I wonder how much of that is from their training corpus and how much is from their baked in personnality.
logicalappeals•54m ago
iamacyborg•49m ago
It seems to be getting distinctly dumber and pulling more and more irrelevant context from historical conversations.
mdp2021•44m ago
Well, we need Intelligence (Pandora's box is open, now we need the Real Thing urgently). Typical (aggregate) positions, dumb as expected, will be overcome by a Reasoner. (And I can say, already a number of LLMs can reason even when they start from cretinous aggregate positions if you give them the proper freedom of assessment.)
insanitybit•43m ago
Codex doesn't have any of the annoying "personality" quirks, or at least they haven't gotten worse in the last year whereas Opus 4.6 was the last Anthropic model before things started to get actively worse (not any better at coding, strictly more annoying to have a discussion with).
nolok•16m ago
I agree with the general idea though in not so much detail as you, but I would add that the personality they're giving it is not one of a good teacher or guide, but instead one of an arrogant know it all. That's why it creates problem.
I have no problem with my AI telling me no you're wrong and explaining to me why with details and sources and everything. I actively want that. I know a lot of people can't take that, but that's their loss, they can't take it from human too. But the "you're wrong because you disagree with me" attitude that you need to play around (aka waste time to prove it that IT is wrong not you, and then it just say "oh yeah" and goes on) is one hell of a pain in the ass I'm starting to be tired off.
Gemini might be wrong all the time and absuredly unreliable for anything that's not consensus or adversorial based, but at least it freaking apologizes.
embedding-shape•2m ago
Planktonne•28m ago
cm2012•15m ago
landl0rd•26m ago
mdp2021•19m ago
The first was when they most obviously acritically repeated what they heard, "hearsay machines", "stochastic parrots". Intelligence requires assessment over every provisional output - a continuous cycle of criticisms over intuition.
The second is proposing doctrinal biases, again without verification of the content - "hysterical reactive machines".
embedding-shape•21m ago
That sounds absolutely bananas and would be reason for me to drop the service yesterday. For curiosities sake, what was the word and if I may ask (unless it's confidential or whatever), could you share the session itself? On the surface it sounds like a bug, as I'm regularly using kind of "vulgar" language (and some projects I work on with agents are NSFW) and never had anything like this happen, even with Claude, although I mostly do use ChatGPT/Codex on a day-to-day basis.
gibspaulding•9m ago
GPT seems to be designed more as a tool. If you want your agent to do what you say without questions and without having its own ideas and agendas you’ll likely prefer it.
Claude on the other hand feels more like an attempt at creating a digital person. If you want a collaborator who will debate with you and come up with its own suggestions for what needs done, you’ll prefer it.
Both companies have shifted around this spectrum from model to model, but lately it feels like they’re moving in opposite directions. It will be interesting to see if one or the other approach ends up winning out in the long run or if the split will continue or even widen.
embedding-shape•4m ago