Why do men go to Hooters? This hardly seems like an academic question.
How about: “An Ethnography of Breastaurant Masculinity: Themes of Objectification, Sexual Conquest, Male Control, and Masculine Toughness in a Sexually Objectifying Restaurant?” That has a certain scholarly ring.
The latter was the title of one of several papers published in credible journals over the past year, but were revealed to be a hoax earlier this month. Others include a discussion of canine rape culture at a dog park; a proposal of a theory that encouraging men to anally self-penetrate would combat transphobia; and a paper on “Solidarity Feminism as an Intersectional Reply to Neoliberal and Choice Feminism” that replaced the anti-Semitic phrases in Hitler’s Mein Kamf with feminist buzzwords. In total, seven of the 20 false papers the hoaxters submitted were accepted by peer-reviewed journals.
The group behind the hoax—Helen Pluckrose, editor of the online magazine Aero, James Lindsay, an author and mathematician, and Peter Boghossian, a philosophy professor at Portland State University—wrote in Aero that they conducted the scheme to expose unscientific ideologies in areas of academia they call “grievance studies.” Fields of cultural and identity studies are overly invested in revealing “power imbalances and oppression” rooted in gender, race, and sexuality, they claim, at the expense of examining the evidence. The hoax, they said, proved they were right.
Others disagreed. The “Sokal Squared” hoax, as it’s been termed (named after an earlier hoax by Alan Sokal), has led to a fierce debate between those who believe the pranked journals should be embarrassed, and those condemning the hoaxters themselves. The Chronicle of Higher Education published a collection of responses from academics this week, with such conflicting headlines as “In Defense of Hoaxes” and “A Hollow Exercise in Mean-Spirited Mockery.”
And Twitter is replete with threads making the case that the hoax exposes the sloppy standards of “grievance studies”…
…as well as those arguing that, actually, the hoaxters overstated their achievements.
How can the scam be insignificant given that, as the hoaxters write, they “managed to get seven shoddy, absurd, unethical and politically-biased papers into respectable journals in the fields of grievance studies”?
Though the journals are all peer-reviewed, critics of the hoax have pointed out that some, such as the Journal of Poetry Therapy, are pretty fringe. Meanwhile, an article published in one of the most respectable journals, Hypatia, is far from outlandish. “Most of the 20-page paper is a reasonable synthesis of others’ ideas about oppression and humor,” Justin Weinberg, philosophy professor at the University of South Carolina, wrote on his blog Daily Nous. “It may not be groundbreaking (as one of the reviewers points out), but it is not ridiculous.”
The hoaxters may have exaggerated the scale of their success, but, according to critics, that’s not their worse offense. The biggest criticism against the experiment is that it didn’t adhere to the basic principles of decent science. The hoax didn’t effectively show anything about “grievance studies” because it didn’t have a control group. They showed that peer-reviewed journals do accept fraudulent papers, yes, but this says nothing about whether “grievance studies” journals are particularly inclined to sloppy scholarship, because the hoaxters didn’t even try to get false papers into the journals of other fields.
Had they tried, they might well have been successful. As Twitter academics rushed to point out, journals in presumably more respectable fields have published countless poorly reasoned papers and many that have ultimately been proven false and retracted. “Bad reasoning, unfalsifiable claims, statistical rituals, ideologically driven work, politicking, etc…is a MAJOR concern in psychology, neuroscience, economics, medicine, biology, and many other fields, which the ‘hoax’ authors seem much less concerned about,” wrote Brian Earp, research fellow at the Oxford University Uehiro Center for Practical Ethics.
In recent years, psychologists have attempted to repeat famous experiments but failed to produce the same results as the originals, revealing that many seemingly credible academic claims are in fact highly tenuous. These findings have been used to argue that science has a “replication crisis.” As a 2012 paper “The rules of the game called psychological science” chronicled, psychologists use various methods to inflate their findings. The replication crisis shows these methods dilute the veracity of published work. Other fields have fared little better; for example peer review, the supposed gatekeeper for bad science, has been shown to be ineffective at identifying high-quality papers in medicine.
Economics, another seemingly “hard” field, has also had its share of weak papers.
In 2010, academics published a paper showing that average real economic growth slows by 0.1% when a country’s debt is more than 90% of gross domestic product. This “fact” was publicized at a time when politicians around the world were implementing austerity measures, and was used to help justify the policies. As Tim Fernholz reported in Quartz, the finding was discussed in the US senate at the time and cited by European commissioner Olli Rehn; Norman Lamont, a member of the UK House of Lords and an adviser to former UK chancellor George Osborne; and former US treasury secretary Tim Geithner, among others. But, three years later, academics discovered that the statistic was based on three Excel spreadsheet errors and, actually, economic growth increases by 2.2% in times of such debt.
Within this context, it’s clear what the recent hoax showed was not that “grievance studies” is a weak field—but that all research, even papers published in peer-reviewed journals, should be approached with some skepticism.
Single studies do not definitively “prove” anything, and even papers published in the best journals are not unquestionable evidence. Good theories are built up slowly, over time, based on multiple papers and large studies. Ultimately, there’s no shortcut to figuring out the truth.