deterministic LLMs - what are they good for? do we need them? someone explain ...
Comments
handfuloflight•7mo ago
Prompt: "Output a name of one of the states of the United States."
It will not tell me the names of the planets. Or the names of prescription drugs.
Why is this level of determinism not enough?
theGeatZhopa•7mo ago
at some point one might need determined responses. RAG & Law, or in certain situations, where RAG is done on f.e. legal or some constraints are in hold. LLMs shouldn't respond non-deterministic then.
But this is solveable without deterministic AI. Nonetheless, to have some deterministic AI would solve some pain in some areas you & me not work in, but others.. ;)
handfuloflight•7mo ago
I also work in those situations and I don't see how prompting with a grounded context isn't giving the determinism needed?
theGeatZhopa•7mo ago
may be your needing of determinism isn't everyones needing of determinism and you never worked on those situations where prompting with a grounded context didn't give the determinism needed??
I think, [1], [2] and [3] would be happy to discuss and gladly learn from your experience. That's all about discussion - learn from others & teach some others. I'll follow them and look forward to learn from the discussions and your arguments, too.
rvz•7mo ago
Of course we do.
Reliability is more valuable than anything.
Would you want the AI system to mistake the sun for a yellow warning sign whilst you are driving on the highway?
handfuloflight•7mo ago
It will not tell me the names of the planets. Or the names of prescription drugs.
Why is this level of determinism not enough?
theGeatZhopa•7mo ago
But this is solveable without deterministic AI. Nonetheless, to have some deterministic AI would solve some pain in some areas you & me not work in, but others.. ;)
handfuloflight•7mo ago
theGeatZhopa•7mo ago
may be you should talk to this guys:
[1] SteadyText: Deterministic LLMs: Same input → same output, every time (julep.ai) -> https://news.ycombinator.com/item?id=44458132 (7d ago, 0 comments)
[2] Enhancing LLM performance with reasoning using deterministic feedback loops (usekbai.com) -> https://news.ycombinator.com/item?id=44482032 (4d ago, 0 comments)
[3] The AI Agent schism: deterministic vs. non deterministic (kunle.app) -> https://news.ycombinator.com/item?id=44379994 (7h ago, 0 comments)
I think, [1], [2] and [3] would be happy to discuss and gladly learn from your experience. That's all about discussion - learn from others & teach some others. I'll follow them and look forward to learn from the discussions and your arguments, too.