To me, starting to solve the problem by meticulously measuring it, is a sign of a good solution.
:p
Being able to admit the flaws and limitations of a technology is often critical to advancing adoption. Unfortunately, producers of currently popular learning model based technologies are more interested in speculation and growth and speculative growth than genuinely robust operation. This paper is a symptom of a larger problem that is contributing to the bubble pop, downturn, or "AI winter" that we are collectively heading toward.
knowledgeinfra•1h ago
petesergeant•1h ago
chrisjj•1h ago
This premise is unsound. We don't expect LLMs to deliver with fidelity, just as we don't expect parrots to speak with their owners' accents. So infidelity is by no means a failure.