How effective is AI at pulling out relatively niche data from a document? Like - tell me which of these contracts has an arbitration clause?

Seems fine as long as the startup has a lawyer that makes sure the contracts are not too out of the ordinary. If they also just use AI, their blind spots will overlap with you blind spots and I wouldn't trust the process completely.

even before AI: when i raised from USV the second time, my lawyer and theirs had already negotiated paperwork with another company, so everyone decided to reuse it. it was MUCH cheaper…

I'm a lawyer and I do not do this type of work. My spidey sense is tingling nevertheless, raising questions about UPL and whether this appears to reduce upfront costs in exchange for a time-bomb of (not so) invisible risk and far greater costs down the line. What happens when things go pear-shaped and the claim is lack of due diligence? "Oh, 'the chatbot must have missed that,' you say?" A single lawsuit from anywhere could easily cost multiple times what the firm quoted.

An ounce of prevention is worth a pound of cure.