This paper puts words to something I’ve noticed repeatedly with LLMs, particularly Qwen3.6. When I read its reasoning, it appears to recognise the mistake and then carry on as if it hadn’t noticed it at all.
> models often determine their answers based on implicit biases tied to question templates, then construct reasoning chains to justify their predetermined conclusions
> its reasoning was correct right until the final step (Yes/No answer)
Georgelemental•33m ago
Natural intelligences do this too
freejazz•7m ago
Yeah and it's not great then either
tyg13•5m ago
Must we always see this restated every time? It's getting a bit stale always seeing these kinds of comments on articles about LLM.
kibwen•35m ago
"Study: Communing With The Gods of Mount Olympus Via the Oracle at Delphi Is Not Always Faithful"
florianherrengt•1h ago
> models often determine their answers based on implicit biases tied to question templates, then construct reasoning chains to justify their predetermined conclusions > its reasoning was correct right until the final step (Yes/No answer)
Georgelemental•33m ago
freejazz•7m ago
tyg13•5m ago