Abstract, for those not interested in reading TFA:
"Large language models now write papers and referee reports in economics. The field must develop best practices for handling AI-generated text in the publication process. Otherwise, peer review is in serious danger of collapsing. We propose that AI detectors should become a standard tool of the profession. Our experiment shows that they flag referee reports written by AI, not reports that AI merely proofread or helped edit.
Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake, and that an AI-generated referee report adds nothing that editors could not get by running the prompt themselves. Such reports are unacceptable: they damage authors and journals and should be treated like plagiarism. Individual restraint has already shown its limits, and the profession needs to develop norms to solve this collective action problem. We close with a practical guide for authors, referees, and editors."
p-e-w•5m ago
> Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake
“We propose a witch hunt fueled by unreliable, unaccountable, proprietary detection technology that’s roughly equivalent to tea leaf reading and can be fooled by trivial rewrite templates that anyone can find with a 5-second web search.”
Eddy_Viscosity2•12m ago
Peer review will just be done by other AI, which is technically appropriate because they are peers to the AI authors.
avaer•11m ago
> We are happy to acknowledge that we used LLMs to help with this paper.
Framed another way: AIs are arguing with AIs about the merits of AIs, while the Board of Governors of the Federal Reserve System is nodding.
patrickmay•12m ago
"Large language models now write papers and referee reports in economics. The field must develop best practices for handling AI-generated text in the publication process. Otherwise, peer review is in serious danger of collapsing. We propose that AI detectors should become a standard tool of the profession. Our experiment shows that they flag referee reports written by AI, not reports that AI merely proofread or helped edit.
Aided by a simple model, we argue that a paper flagged as AI-generated should trigger a closer reading and can be rejected at the first mistake, and that an AI-generated referee report adds nothing that editors could not get by running the prompt themselves. Such reports are unacceptable: they damage authors and journals and should be treated like plagiarism. Individual restraint has already shown its limits, and the profession needs to develop norms to solve this collective action problem. We close with a practical guide for authors, referees, and editors."
p-e-w•5m ago
“We propose a witch hunt fueled by unreliable, unaccountable, proprietary detection technology that’s roughly equivalent to tea leaf reading and can be fooled by trivial rewrite templates that anyone can find with a 5-second web search.”