It’s surprisingly easy to game scientific publishing’s most important method

Ivan Oransky giving a Ted talk in 2012.
Ivan Oransky giving a Ted talk in 2012.
Image: TED
We may earn a commission from links on this page.

One of the most important ways to examine the credibility of scientific findings just took a serious hit with the recent revelation of a record-setting peer review fraud at a cancer-research journal.

On April 20, Tumor Biology announced it was retracting more than 100 papers published between 2012 and 2016 over issues with the peer-review process, amounting to nearly one-fifth of the 450 papers retracted for that reason in the same period, according to US-based blog Retraction Watch.

Many of these retractions, including the most recent round, have been for research from Chinese scientists, who often rely on third-party companies to help translate and submit their work to journals. In the case of Tumor Biology, owned by an academic society and published by Springer Nature, external editors asked that submissions include recommendations of people in their field as reviewers and email addresses, a not-unusual practice. However, the email addresses supplied didn’t belong to the suggested reviewers.

Instead, the third-party companies provided fake emails and reviews, according to Springer’s investigation (link in Chinese). The fake reviews came to light as a result of Springer’s investigations resulting from earlier retractions and the publisher recommended new practices; in January, the journal saw new peer-review practices put in place when Tumor Biology became part of SAGE.

Quartz spoke to Ivan Oransky, co-founder of Retraction Watch, about the scale of the problem, how the ubiquity of personal email has made it worse, and how to fix it. A lightly edited transcript follows.

What’s the mechanism behind this scandal?

What appears to happen here is that companies that advertised services for authors and researchers were doing things that were clearly unethical and even illegal, but they were taking advantages of some vulnerabilities in the peer-review system.

It’s not clear that the authors actually knew what was happening, but somehow these companies were able to submit fake email addresses and instead of the people who should have been reviewing the papers, it was somebody within the circle, or the authors themselves, in which case they would have known. But this is a… problem that we first described it in 2012, and wrote a big piece about it in 2014, after which a lot of publishers took notice and tried to make some changes.

Are there any telltale signs to help journals spot problem peer reviews?

One of the reasons that Moon [Moon Hyung-In was a South Korean plant compound researcher who faked email addresses so he could review his own studies] was caught in 2012 was that his peer review came back very quickly, which is very unusual in science. People are busy. Usually journals have to really beg and remind people to send back their reviews. [But in Moon’s case] all of these came back very quickly, some within 24 hours. That makes people suspicious. But I don’t think people are doing that anymore… when you announce to the world this is how we found something, they are going to find a way to get around that.

Most of these papers are legitimate papers, not fraudulent. They went through the process and they look like any other paper. What happened was on the back end in terms of the fake emails. One of the things which is difficult in many countries, including China, is to only accept institutional email addresses. China, in particular, now has a lot of people using 163.com [owned by China’s online portal NetEase] and different email addresses. It’s unfair to assume [that] just because someone uses a personal email address, they are not a legitimate person. One of the things that the journals said they were going to do was to actually check the email addresses.

If someone offers your name as a peer reviewer, as an editor, I should look into it and make sure it is your email address. If it looks strange, I should email you and say, are you really [the reviewer]?

In China’s case, most researchers are writing in English, their second language. Does the language barrier here contribute to the problem?

It’s certainly true that English is the dominant language in terms of science and publication. It’s actually perfectly acceptable for researchers to hire translators or people who format papers. There’s nothing wrong with that. Universities, even here in the States, have people on staff to do those things. The problem is that in any system anywhere, there will be unscrupulous people. So there are unscrupulous people who own these companies who have decided to take advantage of the vulnerability as we discussed in the system. 

But it’s really important to put these numbers in context. For example, not all of the cases of fake peer review over the years have been from China, we’ve seen cases from Iran, Pakistan and South Korea and some other countries.

Retractions are still very rare. Overall there are probably 800 to 900 retractions through a year out of… close to two million papers published per year.

Is this an Asian problem?

The US has the most retractions [for different reasons]. It also publishes the most papers. So I think it’s important to put into context that this [peer review issue] is one reason for retraction. Four hundred and fifty retractions is an impressive number over the last five years but over the five years there were a few thousands of corrections.

It’s very important to understand that scientific fraud and misconduct, retractions are a global problem. 

In China, there are particular incentives and a lot of competition that make things different. As far as I know, there is still this cash incentive and bonus for researchers in China who publish in journals that have a certain impact factor [a measure of the average number of annual citations of a journal’s articles.]

What happened if these papers are cited by other researchers, before being retracted?

Firstly, none of these [107] retracted papers were cited very often. That probably speaks to the fact that these were not huge, important papers. In this case, Tumor Biology is not a very prestigious journal, there weren’t many citations. Secondly, it’s fine to cite a retracted paper as long as you noted it’s retracted. One of the interesting things is that these seem like legitimate papers. The authors should probably resubmit them and have them properly peer reviewed.

Should they declare they were retracted before re-submitting for publication?

I think it’s the honest thing to do, but there’s no laws about that. Especially if they hired a company, and they thought those companies were helping them translate and format the papers, but it turned out the companies have deceived them, well, then I would tell the editors the situation and re-submit it.

How should the peer review system make progress?

I actually believe very strongly in peer review. But I think that it is too limited in its current form. We should be thinking about peer review not just before publication but after. I think we should be requiring researchers to deposit data so that it can be checked. I think that we want to make peer review more inclusive. It’s still skewed in particular ways and time-intensive. A lot of people also talk about whether it should be open. People should sign their reviews and they should be available. There are a lot of pluses and minuses to that system but it’s something that we should look at. A lot of these will require publishers to embrace change.

This post was updated on May 8 with clarifications from Springer Nature on the edit process at Tumor Biology.