frontpage.
newsnewestaskshowjobs

Made with ♥ by @iamnishanth

Open Source @Github

fp.

Open in hackernews

Half of social-science studies fail replication test in years-long project

https://www.nature.com/articles/d41586-026-00955-5
6•MBCook•1h ago

Comments

BeetleB•1h ago
> When some of SCORE’s team members attempted to reproduce the data analyses of 600 papers, they found that only 145 contained enough details to do so.

...

> However, many of the failures might have been caused by the SCORE researchers needing to make guesses about procedures or to recreate raw data, Errington says.

Incidentally, this is not unique to the social sciences. Back when I did condensed matter physics, the same was true for both experimental and theoretical papers. The PIs were pretty open about it: Once they perfect a technique, they want to milk it for as many papers as possible. If they reveal too many details, others will be able to outpublish them.

(Yes, it sucked).

h45x1•46m ago
I am surprised it's just half. The incentives are not there for replication or verification. You need to publish and it is easier to publish something new than to fight over existing results, especially if you're a young researcher.

I maintain something akin to google trends but for abstracts in economics papers. The word "novel" is growing rapidly, while "reproduce", "replicate", and "verify" have not changed much over the years: https://dubovik.eu/blog/repec?t=replicate&t=reproduce&t=veri...