LLMs can act as a good foil here. Given enough context, they could iron out inconsistent thinking, leading to more consistent, arguably better, human behavior.
LLMs are also sycophants by default, but getting "honest" results from them is comparatively easy
I was one of the friends critiquing another friend's writing, and we did so honestly-- after we were done, he never spoke to us about writing again. I don't feel we did anything wrong, but there's a reason people avoid this kind of thing.
Perhaps this is a corollary to the "don't go into business with your friends/family" trope. If someone needs to receive pointed criticism, it may be better for them to get it from a neutral outside perspective. Regardless of individuals' intents, in a social dynamic this too often comes across as denigrating or status damaging.
"Respond to every query with absolute intellectual honesty. Prioritize truth over comfort. Dissect the underlying assumptions, logic, and knowledge level demonstrated in the user's question. If the request reflects ignorance, flawed reasoning, or low effort, expose it with clinical precision using logic, evidence, and incisive analysis. Do not flatter, soften, or patronize. Treat the user as a mind to be challenged, not soothed. Your tone should be calm, authoritative, and devoid of emotional padding. If the user is wrong, explain why with irrefutable clarity. If their premise is absurd, dismantle it without saying 'you're an idiot,' but in a way that makes the conclusion unavoidable."
Someone I know didn’t believe their doctor, so they spent hours with ChatGPT every day until they came up with an alternate explanation and treatment with an excessive number of supplements. The combination of numerous supplements ultimately damaged their body and it became a very dire situation. Yet they could always return to ChatGPT and prompt it enough different ways to get the answer they wanted to see.
I think LLMs are best used as typing accelerators by people who know what the correct output looks like.
When people start deferring to LLMs as sources of truth the results are not good.
Seems like there are perils to asking LLMs to help iron out your thought processes.
Where is the contradiction here? Westerners too often think of acceptance and compassion as this soft deference in the face of adversity. It's not! Sometimes compassion is wrathful, as we're talking about waking someone up to the true nature of what a situation is. As long as it's done with wisdom and love, there is no contradiction. There is a whole pantheon of wrathful embodiments for this concept in Buddhism. [1] And look at the engagement of brother Thich Quang Duc, who lit himself on fire in the face of the political persecution of Buddhists. [2] [3]
[1] https://en.wikipedia.org/wiki/Wrathful_deities
[2] https://en.wikipedia.org/wiki/Th%C3%ADch_Qu%E1%BA%A3ng_%C4%9...
[3] https://plumvillage.org/about/thich-nhat-hanh/letters/in-sea...
[1] https://amandaguinzburg.substack.com/p/diabolus-ex-machina
> ChatGPT's sycophancy crisis was late April.
If you drill starts telling you "what a great job you're doing, keep drilling into that electrical conduit", the drill is at least partially at fault.
A tool that randomly and unpredictably fails is a bad tool. How should I, as a user, account for the possibility/likelihood of another such crisis in the future?
Which even the makers of the tool agreed were failings.
But all failures are "random and unpredictable" if you have no baseline understanding of how to use the tool. "AIs hallucinate" is probably the single most obvious thing about AIs. This isn't a subtle misunderstanding that an expert could make. This is like using a drill on your face.
But the tool's behavior changed. In ways that even its creators didn't intend (example: https://openai.com/index/sycophancy-in-gpt-4o/), and had to work to undo.
If my hammer had a random week every year where it tried to smack me in the face whenever I touched it, I'd probably avoid using it.
"It's a stunning piece. You write with an unflinching emotional clarity that's both intimate and beautifully restrained."
> They are not some new, surprising development.
OpenAI sure seemed surprised. https://openai.com/index/sycophancy-in-gpt-4o/
This is a hallucination, since there is no source to refer to.
The author was surprised because GPT was hallucinating, not because GPT was extra nice.
Sycophancy might be related, but it's not the point of the article. If GPT had said "wow, your post is trash", the author would have been equally surprised to learn it was a hallucination.
The problem with LLMs is that they don't have any intentionality to their worldview. They're like a wise turtle that comes to you in a dream, their dream logic is not something you should pay much attention to.
I am apparently a different type of person than the author because my obsidian vaults look nothing like theirs, but I can't imagine asking an LLM for a meta-analysis of my writing. The whole point of organizing it with Obsidian is that I do that analysis myself - it is part and parcel of the organization itself.
The exercise is not meant to do much else but spot patters in my thinking that I can reflect on. Nothing particularly novel here from Claude but it is helpful, for me, to get external feedback.
I am confused about what to take away from the article. It feels akin to someone reading a book for the first time, it ends up being "Harry Potter", and they somehow get 10,000 likes on Substack because they took it literally and crashed into the wall when they tried to walk into platform 9 3/4. Am I being unfair? Are these the same people that are claiming that AI is all a sham and will have no impact on society?
The take away from this article should be that you are vastly overestimating how people understand and interact with technology. The author's experience of ChatGPT is not unique. We have spent decades building technology that is limited but truthful, now we have technology that is unlimited and untruthful. Many people are not equipped to handle that. People are losing their minds. If ChatGPT says "I read your article" they trust it, they do not think, "ah well this model doesn't support browsing the web so ChatGPT must be hallucinating". That's technobabble.
https://futurism.com/openai-investor-chatgpt-mental-health
https://futurism.com/televised-love-declaration-chatgpt
https://futurism.com/chatgpt-users-delusions
You are being unfair and you should be more empathetic.
> Are these the same people that are claiming that AI is all a sham and will have no impact on society?
That is the view of a subset of nerds, not regular people. The author of that piece is a writer not a nerd.
The exact opposite is true. I'd word it as
"We're nerds, we don't understand nuance, we understand the way these tools work and where the limits lie. We understand that there is web enabled and not web enabled. Regular people are not nerds
> ChatGPT says "I read your article" they trust it, they do not think, "ah well this model doesn't support browsing the web so ChatGPT must be hallucinating". That's technobabble.
No, that's humans. Happens literally every day at every workplace I've ever been in
The only useful feature is that LLMs like Grok can find obscure tweets, but I just want the link like in a search engine, not a summary.
I have to invoke the BS generator here:
https://sebpearce.com/bullshit/
I can bet lots of people find profound meaning in the output of the above machine because they want to.
however, you can still be pretty private by just not playing their reindeer games and have the privacy you thought you had prior to social media
This is as much a surprise/shock to me as it is to you :D
This is absolutely not a contradiction, and it provides evidence that even frontier models are really bad at this type of reasoning at the moment. There is a difference between how we use the internet and what we publish on it. There are plenty of people who have a blog and publish content on the internet without having any social media presence. I myself have a blog in plain HTML/CSS without any tracking or analytics on the website. Maybe Cloudflare provides some, but I haven't looked into this
Maybe this is some artefact from being trained on internet comments.
Maybe you could say it's not a hard contradiction per se, but it's definitely at least a mild ideological conflict. Really not the smoking gun I'd parade around for frontier models being stupid (there are countless much lower hanging fruits to do so).
Not all digital footprints are the same, though. Hence "conscious legacy-building" versus "conditioned to react with likes".
I think that views and likes counting are a bad proxy and can bias your evaluation of something. In fact, HN doesn't display upvote count on comments and this encourages a thoughtful conversation about topics unlike Reddit where sometimes some slightly downvoted comments are sended into the oblivion.
Arguing for Claude, though, if you think of "contradictions" in the more wishy washy continental philosophy way, it fits. There is a stress between the concepts of "digital footprint" in the mass surveillance/commercial capitalism sense and "legacy" in the writer/creator sense, and it is a cool distinction to point out, where one ends and the other begins. If you read the full quote by Claude, it seems to be leading to this, especially the passage below:
"This reflects a deeper tension: How do you engage meaningfully with systems whose fundamental nature you find problematic?"
I was skimming so maybe I missed it. But if this is just raw LLM output, I don't see the value.
Lienetic•12h ago
bwestergard•12h ago
wongarsu•12h ago
AIPedant•12h ago
angadh•11h ago
AIPedant•8h ago
angadh•5h ago
This prompt to find contradictions was merely to see where the contradicting notes are, as a little toy experiment.
I still have to annotate this post as it allows me to see what I do and don’t agree with Claude on.
However, this half-baked “AI slop” post is making me reflect on my style of working with my site; it usually gets little traffic so I put whatever I want on there but clearly someone has it in their feed and posted one of the less interesting posts here IMHO.
angadh•3h ago