frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

Open in hackernews

Should LLMs ask "Is this real or fiction?" before replying to suicidal thoughts?

2•ParityMind•4h ago
I’m a regular user of tools like ChatGPT and Grok — not a developer, but someone who’s been thinking about how these systems respond to users in emotional distress.

In some cases, like when someone says they’ve lost their job and don’t see the point of life anymore, the chatbot will still give neutral facts — like a list of bridge heights. That’s not neutral when someone’s in crisis.

I'm proposing a lightweight solution that doesn’t involve censorship or therapy — just some situational awareness:

Ask the user: “Is this a fictional story or something you're really experiencing?”

If distress is detected, avoid risky info (methods, heights, etc.), and shift to grounding language

Optionally offer calming content (e.g., ocean breeze, rain on a cabin roof, etc.)

I used ChatGPT to help structure this idea clearly, but the reasoning and concern are mine. The full write-up is here: https://gist.github.com/ParityMind/dcd68384cbd7075ac63715ef579392c9

Would love to hear what devs and alignment researchers think. Is anything like this already being tested?

Comments

ParityMind•4h ago
Happy to answer questions or refine the idea — I used ChatGPT to help structure it, but the concern and proposal are mine. Not a dev, just someone who's seen how people open up to AI in vulnerable moments and wanted to suggest a non-therapeutic safety net.
reify•2h ago
you people trying to build these types of apps have not got a fucking clue!

Do you have a degree in psychology, counselling psychology, clinical psychology, psychotherapy, psychoanalysis, psychiatry? anything to do with the care professions?

If not. why are you fucking about in my profession which you know nothing about. Its like me writing few 10 line bash scripts, and them saying I am going to build the next google from home on my laptop.

This is the sort of care real professionals provide to those in crisis in the middle of suicical ideation. It is a crisis.

Every year in the month before Christmas, Therapists who worked in the psychotherapy service, where I worked fo 20 years, had to attend a meeting.

This meeting was to bring any clients who they felt were a suicide risk over the christmas period.

If a client met those criteria, a plan was put in place to support that person. This might mean; daily phone calls, daily meetings, and other interventions to keep that person safe.

Human stuff that no computer program can duplicate.

The Christmas period is the most critical time for suicides, It is the period when most suicides occur.

what the fuck do you fucking idiots think you are doing??

Offering a service, not a service, a poxy app, that has absolutely no oversight, no moral or ethical considerations, which, in my opinion would drive people to suicide completion.

you are a danger to everyone who suffers mental illness.

Thinking you can make a poxy chatgpt app in 5 minutes to manage those in great despair and in the middle of suicidal ideation is incredibly naive and stupid. In therapy terms, Incongruent, comes to mind,

How will you know if those superficial sounds like "ocean breeze, rain on a cabin roof" are not triggers for that person to attempt suicide. I suppose you will rely on some shit chatgpt vibe fantasy coding shit.

This too is absolute bullshit: "Ask the user, They are not users! they are human beings in crisis!:

“Is this a fictional story or something you're really experiencing?” The hidden meaning behind this question is: "Are you lying to me", "Have you been lying to me".

A fictional story is one made up, imaginary. To then ask in the same sentence if that story is real is contradictory and confusing.

Are you assuming that this person has some sort of psychosis and is hearing voices, when you say "something you're really experiencing? Are you qualified to diagnose psychotic or Schizophrenic disorders? How do you know if the psychosis is not a response to illicit drugs.

so many things to take into consideration that a bit if vibe coding cannot provide.

No therapist would ever ask this banal question. We would have spent a long to time developing trust. A therapist will have a taken a full history of the client, a risk assessment, would be fully aware of the clients triggers, and will know the clients back story.

suicide is not something you can prevent with an app.

YES! I do have the right to be angry and express it as I feel fit, especially if it stops people from abusing those who need care. A bystander I am not.

Ask HN: Anybody still into the "quantified self" thing in 2025?

1•znpy•32s ago•0 comments

Measuring power network frequency using junk you have in your closet

https://halcy.de/blog/2025/02/09/measuring-power-network-frequency-using-junk-you-have-in-your-closet/
1•zdw•39s ago•0 comments

A universal interface connecting you to today's AI models

https://tenzorro.com/en/models
1•paulo20223•1m ago•0 comments

Pen Scraping Challenge – Test Our ML Bot Detection on Aegilock

https://www.aegilock.de/
1•TimTom89•1m ago•1 comments

AI Tools Are Not Time Machines

https://trunk.io/blog/ai-tools-are-not-time-machines
2•samgutentag•2m ago•0 comments

Next Big Shift in Search: From Product to Infrastructure

https://lsvp.com/stories/next-big-shift-in-search-from-product-to-infrastructure/
1•feel-ix-343•4m ago•0 comments

You Cannot Make Everyone Happy

https://learnerncoffee.wordpress.com/2025/07/08/you-cannot-make-everyone-happy/
1•fzliu•4m ago•0 comments

Gamma Hit $50M ARR and 50M Users with Just 35 People

https://haebom.dev/archive?post=7vgjr4m1neyrq2dwpy86
1•haebom•5m ago•0 comments

Lessons from YouWare's Founder on Building AI-Native Products

https://medium.com/@alexwang_thoughts/insights-from-an-interview-with-youwares-founder-on-ai-startup-3e91bceaa1fb
1•rand_num_gen•6m ago•1 comments

Powering Agentic Observability with the Observe MCP Server

https://www.observeinc.com/blog/powering-agentic-observability-with-the-observe-mcp-server
1•chenjiayuan•9m ago•0 comments

Fuel switches were cut off before Air India Boeing 787 crash

https://text.npr.org/nx-s1-5465063
1•BostonFern•15m ago•1 comments

South China Sea Monster: New Chinese Ekranoplan

http://www.hisutton.com/Chinese-WIG-2025-07.html
1•EA-3167•15m ago•0 comments

GPT 4 agrees that AI is like an asteroid on human race

https://chatgpt.com/share/6871898e-e9b4-8009-8869-556a5779a050
1•zkmon•19m ago•1 comments

Repomix, a tool that packs your entire repository into a single AI-friendly file

https://github.com/yamadashy/repomix
1•consumer451•19m ago•0 comments

Ecopals Virtual Lab AI-based material development

https://www.ecopals.de/en/virtuallab
1•doener•22m ago•0 comments

Fuel switches cut off before Air India crash that killed 260

2•rishikeshs•23m ago•0 comments

The machine that shaped the internet and help us survive AI

https://theconversation.com/the-forgotten-80-year-old-machine-that-shaped-the-internet-and-could-help-us-survive-ai-260839
4•devonnull•24m ago•0 comments

Show HN: Made an app for streamers to overlay live Telemetry for Bambu printers

https://github.com/joetex/bambu-telemetry-stream-overlay
1•JoeOfTexas•29m ago•0 comments

Flow Alone Won't Make You a Writer

https://thereader.mitpress.mit.edu/flow-alone-wont-make-you-a-writer/
3•billybuckwheat•30m ago•0 comments

Does working from home kill company culture?

https://www.economist.com/business/2025/07/07/does-working-from-home-kill-company-culture
1•nmat•30m ago•1 comments

A software conference that advocates for quality

https://bettersoftwareconference.com/
3•leoncaet•34m ago•0 comments

Making 5/8" Sticks: A Cheap(ish) Hack

https://christopherschwarz.substack.com/p/making-58-sticks-a-cheapish-hack
1•crescit_eundo•34m ago•0 comments

Joseph Suarez (E/) on X: "My Advice for Programming and ML" / X

https://twitter.com/jsuarez5341/status/1943692968013025457
1•bilsbie•35m ago•0 comments

'123456' password exposed chats for 64M McDonald's job applicants

https://www.bleepingcomputer.com/news/security/123456-password-exposed-chats-for-64-million-mcdonalds-job-applicants/
3•nan60•36m ago•0 comments

Mathematicians reveal factors driving gun sales in America

https://phys.org/news/2025-07-mathematicians-reveal-factors-gun-sales.html
1•jshprentz•38m ago•1 comments

Apple Watch Ultra rumored to get high blood pressure detection

https://9to5mac.com/2025/07/11/apple-watch-ultra-3-three-new-features-coming-next-year/
2•brandonb•38m ago•0 comments

How You Loose 51B

https://hugston.com/articles/The_ranking_game
1•trilogic•38m ago•0 comments

The First Trading Copilot

https://www.cotraderapp.com
1•philoinvestor•43m ago•2 comments

TSMC to cease GAN foundry production in 2027 due to Chinese competition

https://www.semiconductor-today.com/news_items/2025/jul/tsmc-030725.shtml
1•ilamont•46m ago•0 comments

LLMs for coding (+ free workflow templates)

https://blog.n8n.io/best-llm-for-coding/
1•based2•47m ago•0 comments