…as well as those arguing that, actually, the hoaxters overstated their achievements.

How can the scam be insignificant given that, as the hoaxters write, they “managed to get seven shoddy, absurd, unethical and politically-biased papers into respectable journals in the fields of grievance studies”?

Though the journals are all peer-reviewed, critics of the hoax have pointed out that some, such as the Journal of Poetry Therapy, are pretty fringe. Meanwhile, an article published in one of the most respectable journals, Hypatia, is far from outlandish. “Most of the 20-page paper is a reasonable synthesis of others’ ideas about oppression and humor,” Justin Weinberg, philosophy professor at the University of South Carolina, wrote on his blog Daily Nous. “It may not be groundbreaking (as one of the reviewers points out), but it is not ridiculous.”

The hoaxters may have exaggerated the scale of their success, but, according to critics, that’s not their worse offense. The biggest criticism against the experiment is that it didn’t adhere to the basic principles of decent science. The hoax didn’t effectively show anything about “grievance studies” because it didn’t have a control group. They showed that peer-reviewed journals do accept fraudulent papers, yes, but this says nothing about whether “grievance studies” journals are particularly inclined to sloppy scholarship, because the hoaxters didn’t even try to get false papers into the journals of other fields.

Had they tried, they might well have been successful. As Twitter academics rushed to point out, journals in presumably more respectable fields have published countless poorly reasoned papers and many that have ultimately been proven false and retracted. “Bad reasoning, unfalsifiable claims, statistical rituals, ideologically driven work, politicking, etc…is a MAJOR concern in psychology, neuroscience, economics, medicine, biology, and many other fields, which the ‘hoax’ authors seem much less concerned about,” wrote Brian Earp, research fellow at the Oxford University Uehiro Center for Practical Ethics.

In recent years, psychologists have attempted to repeat famous experiments but failed to produce the same results as the originals, revealing that many seemingly credible academic claims are in fact highly tenuous. These findings have been used to argue that science has a “replication crisis.” As a 2012 paper “The rules of the game called psychological science” chronicled, psychologists use various methods to inflate their findings. The replication crisis shows these methods dilute the veracity of published work. Other fields have fared little better; for example peer review, the supposed gatekeeper for bad science, has been shown to be ineffective at identifying high-quality papers in medicine.

Economics, another seemingly “hard” field, has also had its share of weak papers.

In 2010, academics published a paper showing that average real economic growth slows by 0.1% when a country’s debt is more than 90% of gross domestic product. This “fact” was publicized at a time when politicians around the world were implementing austerity measures, and was used to help justify the policies. As Tim Fernholz reported in Quartz, the finding was discussed in the US senate at the time and cited by European commissioner Olli Rehn; Norman Lamont, a member of the UK House of Lords and an adviser to former UK chancellor George Osborne; and former US treasury secretary Tim Geithner, among others. But, three years later, academics discovered that the statistic was based on three Excel spreadsheet errors and, actually, economic growth increases by 2.2% in times of such debt.

Within this context, it’s clear what the recent hoax showed was not that “grievance studies” is a weak field—but that all research, even papers published in peer-reviewed journals, should be approached with some skepticism.

Single studies do not definitively “prove” anything, and even papers published in the best journals are not unquestionable evidence. Good theories are built up slowly, over time, based on multiple papers and large studies. Ultimately, there’s no shortcut to figuring out the truth.

📬 Sign up for the Daily Brief

Our free, fast, and fun briefing on the global economy, delivered every weekday morning.