From January to August of last year, John Bohannon submitted an academic study to 304 peer-reviewed scientific journals. All of the them were open access journals, a newer breed of digital-only academic publications that are free for readers but often charge researchers to publish. Bohannon’s study concerned a molecule, extracted from a lichen, that appeared to show promise as a treatment for cancer. It was accepted for publication by 157 of the journals—slightly over half.
There was only one problem. Bohannon isn’t a scientist; he’s a journalist. And he completely made up the study.
Actually he did more than that. He deliberately inserted unscientific material to test whether or not it would be caught by the journals’ peer reviewers. The “cure for cancer” proposition of the study, for instance, should have seriously raised some eyebrows, though in 157 cases, it did not. A bit subtler, though as much a red flag for scientists, was Bohannon’s claim that the lichen-based molecule could be used as a treatment for humans, though it hadn’t gone through a clinical trial. This, too, was missed by half of the journals. What little feedback Bohannon did receive had more to do with the formatting of his manuscript than the content itself.
Bohannon’s sting operation was not the first of its kind. There are many more examples of fake studies getting published in academic journals. Taken together, they point to serious flaws in academic publishing at large: Journals are too eager to publish surprising studies, and the rigor of peer review is faltering.
Though Bohannon (or his editors at Science, the preeminent American scientific journal where he published the details of his hijinx) blamed open access journals, fake (or at least extremely dubious) studies have also been published in traditional, subscription-based journals.
STING! Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity
In 1996, the mathematician Alan Sokal famously published a piece of utter nonsense in the peer-reviewed, postmodern studies journal Social Text. His attack was focused more on the humanities than academic publishing overall, but it still makes peer review look pretty weak. Here’s an excerpt:
But deep conceptual shifts within twentieth-century science have undermined this Cartesian-Newtonian metaphysics; revisionist studies in the history and philosophy of science have cast further doubt on its credibility; and, most recently, feminist and poststructuralist critiques have demystified the substantive content of mainstream Western scientific practice, revealing the ideology of domination concealed behind the façade of “objectivity.”
STING! Independent, Negative, Canonically Turing Arrows of Equations and Problems in Applied Formal PDE
In 2012, the mathematician Marcie Rathke created a study using a tool called Mathgen, which randomly generates math research papers (all of which are complete hogwash). It was accepted for publishing by the open access journal Advances in Pure Mathematics. The study was never published, though, because the journal wanted to charge a $500 fee.
DUBIOUS STUDY! A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus
In 2011, the journal Science came under fire for publishing a study that, though it wasn’t a sting, offered a conclusion so dramatic that it could have been. The claim was that a bacterium was discovered that could use arsenic in its DNA instead of phosphorus, a radical shift in biologists’ notion of the very building blocks of life. The paper made it through three peer reviewers, all of whom were uncommonly lenient given how revolutionary its claim was, an investigation would later reveal.