frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

Open in hackernews

Reasoning models are risky. Anyone else experiencing this?

4•lebonnnn•12h ago
I'm building interviuu (a job application tool) and have been testing pretty much every LLM model out there for different parts of the product. One thing that's been driving me crazy: reasoning models seem particularly dangerous for business applications that need to go from A to B in a somewhat rigid way.

I wouldn't call it "deterministic output" because that's not really what LLMs do, but there are definitely use cases where you need a certain level of consistency and predictability, you know?

Here's what I keep running into with reasoning models:

During the reasoning process (and I know Anthropic has shown that what we read isn't the "real" reasoning happening), the LLM tends to ignore guardrails and specific instructions I've put in the prompt. The output becomes way more unpredictable than I need it to be.

Sure, I can define the format with JSON schemas (or objects) and that works fine. But the actual content? It's all over the place. Sometimes it follows my business rules perfectly, other times it just doesn't. And there's no clear pattern I can identify.

For example, I need the model to extract specific information from resumes and job posts, then match them according to pretty clear criteria. With regular models, I get consistent behavior most of the time. With reasoning models, it's like they get "creative" during their internal reasoning and decide my rules are more like suggestions.

I've tested almost all of them (from Gemini to DeepSeek) and honestly, none have convinced me for this type of structured business logic. They're incredible for complex problem-solving, but for "follow these specific steps and don't deviate" tasks? Not so much.

Anyone else dealing with this? Am I missing something in my prompting approach, or is this just the trade-off we make with reasoning models? I'm curious if others have found ways to make them more reliable for business applications.

What's been your experience with reasoning models in production?

Comments

techpineapple•11h ago
Don't have an answer to your exact question, but waxing philosophical for a moment, it's interesting that if you were talking to a person they would use their experience to maybe redirect you if they thought your intentions were wrong - subtly or overtly. It's probably really hard, in a casual way to get all the biases right in an LLM. When should it listen to you directly, and when should it say, no your wrong? There are probably a ton of use cases where you want the "wrong" answer, almost anything innovative is going to go against the grain of what a system would think is "right" by definition. Be 77% confident in your answer is hard to tune and a bad UX.
PaulHoule•11h ago
The difference is that in one case you care if the result is right and in another case you don't. If your "complex problem solving" was being used to "design a manufacturing process" or "diagnose a disease and come up with a treatment plan" or something that has consequences if it is wrong you're going to be unhappy.

On the other hand there is a lot of high social-status work that has no consequences like "write a college commencement address" and people will be impressed no matter what comes out.

Perhaps the likes of Sam Altman and Satya Nadella are so excited for AI because it can do their job but it can't fold towels or be the final assembly programmer for software that users actually use.

deepsiml•10h ago
How simple can you break down the process?

So, I did a dream analysis agent setup, and had to break every piece down. One focused on symbols, one focused on memory retrievals, one focused on emotions.

It meant I needed far more agents, but it was a much better result I believe.

Agentic Coding Tools – Not Skynet, Not a Stochastic Parrot

https://www.brethorsting.com/blog/2025/07/agentic-coding-tools-not-skynet,-not-a-stochastic-parrot/
1•aaronbrethorst•2m ago•0 comments

Listen to Rfc2119

https://ericwbailey.website/published/you-must-listen-to-rfc-2119/
1•bluGill•3m ago•1 comments

Armin Ronacher on Agentic Coding

https://www.youtube.com/watch?v=nfOVgz_omlU
1•paulsutter•8m ago•0 comments

Super Simple "Hallucination Traps" to detect interview cheaters

3•EliotHerbst•16m ago•0 comments

A customizable and extensible all-purpose diagrams library for Blazor

https://github.com/Blazor-Diagrams/Blazor.Diagrams
1•mountainview•18m ago•0 comments

Coinbase Acquires LiquiFi

https://www.coinbase.com/es-la/blog/Coinbase-acquires-LiquiFi-the-leading-token-management-platform
1•wslh•19m ago•0 comments

Trans-Taiga Road:The farthest you can get from a town on a road in North America

https://www.jamesbayroad.com/ttr/index.html
2•jason_pomerleau•23m ago•0 comments

Checklist Genie App – Last Call for Beta Testers

https://checklistgenie.app
1•alohaplannerapp•23m ago•0 comments

Show HN: I created a privacy respecting ad blocker for apps

https://www.magiclasso.co/insights/app-ad-blocking/
1•bentocorp•25m ago•0 comments

An Analysis of Links from the White House's "Wire" Website

https://blog.jim-nielsen.com/2025/links-from-whgov-wire/
1•OuterVale•32m ago•0 comments

Why are my Product Hunt upvotes delayed

https://www.ceresai.xyz/
1•Mahsanziak9•41m ago•2 comments

Qualcomm's Centriq 2400 and the Falkor Architecture

https://chipsandcheese.com/p/qualcomms-centriq-2400-and-the-falkor
1•brian_herman•41m ago•0 comments

Bridging Shopify and Shipstation on Heroku: A Story of Custom Fulfillment

https://kevinhq.com/shopify-shipstation-heroku-integration/
1•kevinhq•44m ago•0 comments

My official list of post-glitch.com hosting options

https://livelaugh.blog/posts/glitch-alternatives/
1•raybb•46m ago•1 comments

All high value work is deep work, and all motivation is based on belief

https://www.reddit.com/r/ExperiencedDevs/s/qV1w0XeFPw
2•Crier1002•47m ago•0 comments

'There is a problem': Meta users complain of being shut out of their accounts

https://www.bbc.com/news/articles/cvgnp9ykm3xo
4•mikece•48m ago•1 comments

Mount Everest's Trash-Covered Slopes Are Being Cleaned by Drones

https://www.bloomberg.com/news/features/2025-07-03/dji-drones-clean-up-mount-everest-trash-in-record-time-amid-climate-change
2•nharada•50m ago•2 comments

Gaming on a Medical Device [video]

https://www.youtube.com/watch?v=rf-efIZI_Dg
1•JKCalhoun•50m ago•1 comments

Open Source 1.7tb Dataset of What AI Crawlers Are Doing

https://huggingface.co/datasets/lee101/webfiddle-internet-raw-cache-dataset
3•catsanddogsart•57m ago•0 comments

Microsoft will lay off 9k employees, or less than 4% of the company

https://techcrunch.com/2025/07/02/microsoft-will-lay-off-9000-employees-or-less-than-4-of-the-company/
5•mrcsharp•57m ago•2 comments

Whole-genome ancestry of an Old Kingdom Egyptian

https://www.nature.com/articles/s41586-025-09195-5
6•A_D_E_P_T•1h ago•0 comments

NYT to start searching deleted ChatGPT logs after beating OpenAI in court

https://arstechnica.com/tech-policy/2025/07/nyt-to-start-searching-deleted-chatgpt-logs-after-beating-openai-in-court/
6•miles•1h ago•0 comments

AI virtual personality YouTubers, or 'VTubers,' are earning millions

https://www.cnbc.com/2025/07/02/ai-virtual-personality-youtubers-or-vtubers-are-earning-millions.html
3•pseudolus•1h ago•0 comments

US rural communities bearing the brunt of Bitcoin mining

https://www.dw.com/en/us-rural-communities-bearing-the-brunt-of-bitcoin-mining/a-72889383
4•musha68k•1h ago•1 comments

gmailtail: tail -f Your Gmail

https://github.com/c4pt0r/gmailtail
1•c4pt0r•1h ago•0 comments

A Non-Partisan U.S. Military Is Essential

https://time.com/7296041/non-partisan-military-is-essential/
5•herecomethefuzz•1h ago•0 comments

What to build instead of AI agents

https://decodingml.substack.com/p/stop-building-ai-agents
46•giuliomagnifico•1h ago•28 comments

Flint, Michigan replaces most lead pipes 10 years after Michigan water crisis

https://www.nbcnews.com/news/us-news/flint-replaces-lead-pipes-10-years-michigan-water-crisis-rcna216442
5•toomuchtodo•1h ago•2 comments

Nebius emerged from Russia as one of Nvidia's top-performing investments

https://sherwood.news/tech/nebius-nvidia-gpus-ai-startup/
2•gmays•1h ago•0 comments

One Life

https://thisisyouronelife.com/
1•tasshin•1h ago•0 comments