Show HN: AI agents that validate your product idea by talking to real users

https://app.holyshift.ai/ai/project

7•Matzalar•2mo ago

I built a tool to solve a problem I kept running into: I was making product decisions based on guessing instead of real users. I kept building stuff nobody wanted as I was usually wrong.

So, I built HolyShift: AI agents that validate product ideas by talking to real people on Reddit, HN, X, and LinkedIn … then generate a detailed GTM and “Should we build this?” report.

No synthetic data (ChatGPT). No predictions. Only real conversations from real people.

What it does • Posts platform-native questions (where allowed) • Collects real reactions, objections, pricing signals • Clusters feedback into themes (pain, demand, adoption, pricing …) • Runs a monitoring agent for sentiment analysis • Produces a short validation report (PRD + GTM)

All actions are rate limited and reviewed by a human for compliance.

How it works (technicals) • Multi-agent pipeline (intake → landscape → engagement → monitoring → synthesis → report) • Platform specific prompting (HN vs Reddit vs LinkedIn …) • Real-time sentiment + clustering via embeddings

Link https://www.holyshift.ai (Early beta)

What I’m looking for • What should stay human vs automated? Should we automate this 100%? • How do you do your product validation? Do you talk to your potential users (and who?) before you build?

Happy to answer anything.

Comments

lovrok23•2mo ago

I'm curious about the guardrails here. In my experience trying to use LLMs for user research, they tend to be "yes man" often hallucinating features or agreeing to user requests that aren't actually on the roadmap just to keep the conversation flowing.

how do you constrain the agent to stick strictly to the facts of the product hypothesis without making stuff up to please the potential customer?

Matzalar•2mo ago

We ran into the same issue early on. Our fix was to lock each agent to a small JSON snapshot of the idea (no other knowledge), plus strict response templates. They can only ask questions, never describe features or promise anything. If a user asks for something outside scope, the agent replies with “not in the current hypothesis, why is that important to you?” rather than making stuff up. We also have a human review step before anything goes live.

KurSix•2mo ago

If you have human review for every action, then this isn't scalable software, it's consulting. Either you'll eventually remove this step for scale (and get banned by platforms), or your product will have to be very expensive (to pay for the reviewers' time). Maybe it's worth keeping AI only for the analysis part and leaving the communication to humans?

likethejade87•2mo ago

Are agent pitching ideas or do actual research? Sounds super interesting though

Matzalar•2mo ago

They’re not pitching or selling anything they only do research. The agents ask structured questions in relevant communities and collect real reactions, pain points, objections ... No selling, no marketing language.

tene80i•2mo ago

Interesting idea. Nice design. But usability issue: on mobile I hit your yellow chat CTA thinking it was submitting the app text input. You might want to move that out of the way.

Matzalar•2mo ago

Good point. Thanks for your suggestion. We’re very early, a lot is changing as we move on and get more feedback.

thebiggodzzila•2mo ago

Chat always boosts my confidence, but reality isn’t always as kind. How can I really tell if my idea is any good beyond what Chat says, and how many people do you actually interact with?

Matzalar•2mo ago

Chat is OK at making anything sound promising, but it will not talk to real people and give you the real market signals. We do both, we do the “synthetic” data analysis, but more importantly we talk to real people and ask them what they think. Depending on the product (or project), we’ll usually get few hundred real interactions. The goal is to get enough real, unfiltered feedback to see whether there’s a consistent signal to make a decision.

KurSix•2mo ago

The irony is that people give the most honest and unfiltered feedback when no one is asking them. When a bot approaches a user (even a research one), the observer effect kicks in, and answers often become more polite or socially desirable. Analyzing historical threads where people are just complaining to each other often yields a much more accurate and "raw" pain signal than direct questioning. There, people aren't trying to be nice to your agent.

KurSix•2mo ago

The idea of validation on real data is great. But why post questions directly? It's risky and intrusive. It would be much safer and more powerful to simply passively observe. Have agents scan the thousands of existing threads where people are already complaining about problems. You'll get honest data without risking being banned. People have already told you their problems, you just need to (automatically) listen, not ask again

_demo•2mo ago

Cool idea I have seen this people having this same problem so many times in the past.. The biggest risk I see is how do you know if people responding are providing actually usefull feedback or just garbage that will hurt me in the long run?

Show HN: The Codeverse Hub Linux

Take a trip to Japan's Dododo Land, the most irritating place on Earth

British drivers over 70 to face eye tests every three years

BookTalk: A Reading Companion That Captures Your Voice

Is AI "good" yet? – tracking HN's sentiment on AI coding

Show HN: Amdb – Tree-sitter based memory for AI agents (Rust)

OpenClaw Partners with VirusTotal for Skill Security

Show HN: Seedance 2.0 Release

Leisure Suit Larry's Al Lowe on model trains, funny deaths and Disney

Towards Self-Driving Codebases

VCF West: Whirlwind Software Restoration – Guy Fedorkow [video]

Show HN: COGext – A minimalist, open-source system monitor for Chrome (<550KB)

FOSDEM 26 – My Hallway Track Takeaways

Show HN: Env-shelf – Open-source desktop app to manage .env files

Show HN: Almostnode – Run Node.js, Next.js, and Express in the Browser

Dell support (and hardware) is so bad, I almost sued them

Project Pterodactyl: Incremental Architecture

Styling: Search-Text and Other Highlight-Y Pseudo-Elements

Crypto firm accidentally sends $40B in Bitcoin to users

Magnetic fields can change carbon diffusion in steel

Fantasy football that celebrates great games

Show HN: Animalese

StrongDM's AI team build serious software without even looking at the code

John Haugeland on the failure of micro-worlds

Show HN: Velocity - Free/Cheaper Linear Clone but with MCP for agents

Corning Invented a New Fiber-Optic Cable for AI and Landed a $6B Meta Deal [video]

Show HN: XAPIs.dev – Twitter API Alternative at 90% Lower Cost

Near-Instantly Aborting the Worst Pain Imaginable with Psychedelics

Show HN: Nginx-defender – realtime abuse blocking for Nginx

The Super Sharp Blade

Show HN: The Codeverse Hub Linux

Take a trip to Japan's Dododo Land, the most irritating place on Earth

British drivers over 70 to face eye tests every three years

BookTalk: A Reading Companion That Captures Your Voice

Is AI "good" yet? – tracking HN's sentiment on AI coding

Show HN: Amdb – Tree-sitter based memory for AI agents (Rust)

OpenClaw Partners with VirusTotal for Skill Security

Show HN: Seedance 2.0 Release

Leisure Suit Larry's Al Lowe on model trains, funny deaths and Disney

Towards Self-Driving Codebases

VCF West: Whirlwind Software Restoration – Guy Fedorkow [video]

Show HN: COGext – A minimalist, open-source system monitor for Chrome (<550KB)

FOSDEM 26 – My Hallway Track Takeaways

Show HN: Env-shelf – Open-source desktop app to manage .env files

Show HN: Almostnode – Run Node.js, Next.js, and Express in the Browser

Dell support (and hardware) is so bad, I almost sued them

Project Pterodactyl: Incremental Architecture

Styling: Search-Text and Other Highlight-Y Pseudo-Elements

Crypto firm accidentally sends $40B in Bitcoin to users

Magnetic fields can change carbon diffusion in steel

Fantasy football that celebrates great games

Show HN: Animalese

StrongDM's AI team build serious software without even looking at the code

John Haugeland on the failure of micro-worlds

Show HN: Velocity - Free/Cheaper Linear Clone but with MCP for agents

Corning Invented a New Fiber-Optic Cable for AI and Landed a $6B Meta Deal [video]

Show HN: XAPIs.dev – Twitter API Alternative at 90% Lower Cost

Near-Instantly Aborting the Worst Pain Imaginable with Psychedelics

Show HN: Nginx-defender – realtime abuse blocking for Nginx

The Super Sharp Blade

Show HN: AI agents that validate your product idea by talking to real users

Comments