There’s an accurate way to confirm fraud: look for inconsistencies and replicate experiments.

If the fraudsters “fail to replicate” legitimate experiments, ask them for details/proof, and replicate the experiment yourself while providing more details/proof. Either they’re running a different experiment, their details have inconsistencies, or they have unreasonable omissions.

> This is Goodhart's law at scale.

Also, Brandolini's law. And Adam Smith's law of supply and demand. When the ability to produce overwhelms the ability to review or refute, it cheapens the product.

You shouldn't just assume that the inverse would be free from fraud. The incentives for fraud still apply even when the system is not democratized.

Tearing down gatekeeping (i.e. "high standards") in pursuit of maximal inclusivity is just another way of saying "regression to the mean."

The gate has been removed from the signal chain, and now the noise floor is at infinity.

In what way was it was democratised? We're not talking about Substacks and YouTube channels here, we're not even talking about arXiv preprints and the like, we're talking about peer-reviewed journal publications, and that system remains gated in much the same way that it was in the 1980s when it comes to trying to publish in it. If anything this system is the poster child for top-down gatekeeping by the recognised authorities, and it's precisely the value of that official recognition that makes people so desperate to break into it. The major changes seem to have been the easy availability of author publication lists and the advent of publication metrics, not things which have been or were ever meant to be particularly democratising for would-be authors; and an increase in the number of people playing the game, driven to a large extent by increasing participation from developing countries, and hopefully not many people would have the gall to argue for a ban on developing-country participation.

The Library of Babel comparison is too fatalistic imo, even granting that it's maybe just an extreme example. The real world doesn't quite resemble a closed system with no metadata. We can still establish chains of trust.

Whether or not people will build resilient chains is another story, contingent on whether the strength of that chain actually matters to people. It probably doesn't for a lot of people. Boo. But inasmuch as I care, I feel I ought to be free to try and derive a strong signal through the noise.

Maybe we need a journal completely dedicated to replication studies? It would attract a lot of attention I think.

So much of this started with the rise of the peer-review journal cartel, beginning with Pergamon Press in 1951 (coincidentally founded by Ghislaine Maxwell's father). "Peer review" didn't exist before then, science papers and discussion was published openly, and scientists focused on quality not quantity.

Right, it seems that many of the weaknesses in the system exist because they serve the interests of journal publishers or of normal, legitimate-ish researchers, but in the process open the door to full-time system-hackers and pure fraudsters.

Do you want issues of Nature and cell to be replication studies? As a reader even from within the field, im not interested in browsing through negative studies. It'll be great if I can look them up when needed but im not looking forward to email ToC alerts filled with them.

Also who's funding you for replication work? Do you know the pressure you have in tenure track to have a consistent thesis on what you work on?

Literally every single know that designs academia is tuned to not incentivize what you complain about. Its not just journals being picky.

Also the people committing fraud aren't ones who will say "gosh I will replicate things now!" Replicating work is far more difficult than a lot of original work.

I found the article and your third-hand anecdotes troubling. The good news is that it does not match any of the years of experience in my field. Fraud is just not that rampant. At PhD-granting institutions, the level of fraud you describe here is very seriously punished. It's career-ending. The violations that you are serious enough that any institution would expel said students (or harshly punish faculty--probably firing them). She did no one any favors by not reporting them.

Unfortunately I don't think a dialogue around vague anecdotes is going to be particularly enlightening. What matters is culture, but also process--mechanisms and checks--plus consequences. Consequences don't happen if everyone is hush-hush about it and no one wants to be a "rat".

yeah - skeptical here. Among certain departments, at large schools, under certain leaders.. The combination of "my marriage almost crumbled" for motivated reasoning, and "I have never seen any of this before" total inexperience with actual process.. the post shows itself to be biased and unreliable.

However, among certain departments, at large schools, under certain leaders.. yes, and growing

$0.02

>methods are technically challenging.

And finanacially too..

>Science really is self-correcting..

When economy allows it....