Ask HN: What AI feature looked in demos and failed in real usage? Why?
8•kajolshah_bt•2w ago
Everyone has a story. People LOVE dunking on demos.
Comments
rtbruhan00•2w ago
Real-time voice translation looked amazing in demos, but in practice it struggled with accents, technical jargon, and context. The demos were clearly done in controlled environments with clear speakers and simple topics.
The reason? Training data bias and the "last mile" problem - demos use ideal conditions while real usage involves messy audio, overlapping speech, and domain-specific vocabulary the models never saw during training.
kajolshah_bt•2w ago
Totally agree — the “demo vs real world” gap is always the messy edge cases: accents, crosstalk, domain terms, and people talking like… people.
Did you end up adding any guardrails (confidence thresholds, “please repeat,” glossary/term injection, or human fallback)?
Also curious: were failures mostly ASR or translation/context?
pawelduda•2w ago
Some meta demos failed in demos and in real usage
kajolshah_bt•1w ago
Yes, exactly. A lot of demos just don’t fail in the real world. They were never designed for real usage in the first place. They work once, in a clean flow, and fall apart as soon as people behave… like people.
rtbruhan00•2w ago
The reason? Training data bias and the "last mile" problem - demos use ideal conditions while real usage involves messy audio, overlapping speech, and domain-specific vocabulary the models never saw during training.
kajolshah_bt•2w ago
Did you end up adding any guardrails (confidence thresholds, “please repeat,” glossary/term injection, or human fallback)? Also curious: were failures mostly ASR or translation/context?