My tax accountant mis-filed my taxes a couple years ago, which cost me about $20K (which I learned after I switched to another accountant later). Don't know if Claude / GPT would make the same mistake or not given SOTA today, but feels like there's an opportunity for models to be saving me thousands or tens of thousands of dollars a year.

i want to always have a fast way of reviewing my tax returns without having to hassle my accountant or hire another one.

Sharing all my financial info with LLM companies is not something that I like.

EDIT: good point that paying accountant does not ensure avoiding it

wonderful, you made a good point that make me dislike situation even more

I'm surprised Sonnet 5.5 did noticeably better than Opus and Fable. Same pattern in their other benchmarks.

What is human baseline? How it stacks against 10% LLM accuracy?..

I have a non-trivial tax situation, as a US citizen living abroad. I prepare my own tax returns, never having found a cross-border tax preparer I trust, and have for the past few years, fed my draft return along with all the supporting paperwork into Claude. It has:

1. found errors, e.g. where I used the incorrect exchange rate (and managed to work out how I had made that error, which I was impressed by) 2. found items which would appear to be errors except for context it did not know 3. flagged areas where it disagreed with me as to the correct tax treatment, but accepted that the law is ambiguous (and in some cases I preferred its interpretation after additional research and in other cases I did not) 4. flagged areas where it disagreed with IRS guidance, and only acknowledged said guidance after I provided links to it 5. provided highly inaccurate estimated tax calculations, and not been able to fix its estimates even when challenged

So my takeaway is that it is a very useful tool for tax, but perhaps unsurprisingly, it is not good enough to dump all your paperwork into and expect a shiny return you don't need to check. (Possibly it is if you just have a W-2 and no other substantial income, but I can't speak for that scenario.)

How much can someone trust a report about AI from a company who's domain ends in ".ai"

Not much, I assume.

Not related but I hope QuickBooks learn something and make their software better.

I downloaded 1700 of my transactions and use AI to build an app for it, so I can analyze the transactions and make sure that everything is properly categorized. It used to take me whole day because of the Quickbooks. It took me less than an hour to go through all the transactions.