The scary part isn't that the AI hallucinated - it's that the policy went through without anyone fact-checking the source. Makes you wonder how many other decisions are being made based on unverified AI outputs.

Critical reminder: AI is a tool, not a replacement for due diligence.

With AI, there is a strong incentive for the employee to churn out the report quickly and take Friday off, since no one is going to read it anyway. Maybe they still skim the report and make sure it looks reasonable, but they don't catch the smaller details that are wrong, and more importantly, they don't think deeply or critically about the issue at hand since the report looks reasonable at a glance.

"Show me the incentive, and I'll show you the outcome" - Charlie Munger

My guess is that reporting processes will change and adapt once everyone realizes how useless the old system is in an AI world. A long report used to signal "we thought about this, so rest easy". Now, a long report doesn't signal that. It no longer serves the function it once did.