BROOKE GLADSTONE:  When a paper released by a scientific journal turns out to be wrong, either due to human error or intentional fraud, the journal’s editors often will issue a retraction. It turns out retractions are soaring. According to the Wall Street Journal, since 2001 the number of retractions issued by journals has increased 15-fold.

And in the March issue of Infection and Immunity – I can’t believe it took me two months to get to it – editors cried out for reform. A little while back, we asked science writer Jonah Lehrer what he thinks is going on.

JONAH LEHRER:  The explanation I’m most attracted to is simply that there’s been a norm change. In years past, they used to wait for serious evidence of scientific fraud, and often co-authors would have to insist on retraction. Now I think they’re much more willing to retract a paper if there’s simply an error.

BROOKE GLADSTONE:  Can you give me a couple of examples, so we can understand what’s at stake?

JONAH LEHRER:  The kind of retractions that are most worrisome are retractions that very quickly become the basis for policy. So we’re talking here mostly about retractions in clinical trials.

BROOKE GLADSTONE:  So an example might be a kidney drug that everybody thinks works, and it turns out it actually leaves you open to so many other related diseases that it’s dangerous to take.

JONAH LEHRER:  Yeah, it could be a kidney drug, it could be things like hormone replacement therapy. The list goes on and on and on. I mean, you know, I think people are now pushing back against antioxidants, against the beneficial effects of various vitamins. So there are lots and lots of clinical trials that have gotten lots of media attention, and then the retraction is published in the fine print.

I also want to make the point that I don’t think there’s anything necessarily bad about an increase in retraction. I think, if anything, this represents, you know, a new kind of transparency in peer review.

BROOKE GLADSTONE:  You weren’t that bothered by the retraction study but there was a related study that really did worry you.

JONAH LEHRER:  Yes. This was a rather obscure paper published [LAUGHS] in a rather obscure journal called Ecosphere, which looked at a small subset of papers in the fishery sciences. It was done by researchers at the University of Washington, and they were interested in what happens after a paper is refuted or retracted. So let’s say a paper is published in Nature or Science and they looked at papers published in these two very prestigious journals, and then a few years go by and numerous papers have come out and they prove that this paper just isn’t true, that these researchers made a mistake. What happens next? What happens in the trail of scientific citations? Is this falsified paper, is it still cited, or do other scientists recognize the mistake?

And what they discovered is, I think, deeply depressing, if you believe in peer review, because they found that the number of citations for the falsified paper, for the paper which has been soundly refuted don’t decrease and, in fact, that they remain years and years after being refuted about 17 times higher than the refuting papers.

BROOKE GLADSTONE:  Mm-hmm.

JONAH LEHRER:  So it’s as if the mistake has been demonstrated - it just still persists; it’s very tough to get rid of.

BROOKE GLADSTONE:  Well, we know that in the mainstream media, once an impression is made, it’s very hard to unmake. But you would hold fellow scientists to a higher standard. You would think that they would at least keep up with their own research.

JONAH LEHRER:  You would certainly hope so. And, in fact, I think one of the most troubling parts of this paper is they show that even when these papers which are showing that the original paper is wrong are cited, about 10% of the time they’re cited as demonstrating affirmative proof.

BROOKE GLADSTONE:  [LAUGHS]

JONAH LEHRER:  So scientists cite them in the wrong way.

BROOKE GLADSTONE:  So they don’t even read their own citations. Is this because –

   [LEHRER LAUGHS]

- of an overreliance on grad students?

JONAH LEHRER:  [LAUGHS] No. I think one thing that’s happening is just sheer laziness. I think lots of people use citations without fully reading the papers. They look at the title and maybe the abstract. I think one thing that’s happening is that it’s tough to let go of a beloved theory. These papers were very influential, often for up to a decade. They’re published in big name journals, so people took them very seriously. So the mind, for all sorts of reasons, struggles to let go of those ideas.

BROOKE GLADSTONE:  So what do we do about this?

JONAH LEHRER:  One of the very simple fixes is simply when scientists call up a paper that’s been refuted in PubMed or Google Scholar, or one of these search engines that scientists all over the world use, that they should be automatically linked to the refuting papers.

BROOKE GLADSTONE:  Mm-hmm.

JONAH LEHRER:  So, so if a study isn’t true, we should know that it’s not true and the papers that prove that it’s not true should be right there as well, so they’re easy to cite and the record is automatically corrected.

BROOKE GLADSTONE:  On the other hand, you said that even when scientists are aware of the contradictory papers, they either don’t read them or misread them or use them to support the original wrong conclusions.

JONAH LEHRER:  Well, I think we’d have to invent a new format to be some new kind of link, where you’d know that these were citations which weren’t supporting the original paper –

BROOKE GLADSTONE:  [LAUGHS]

JONAH LEHRER:  -but were actually contradicting it.

BROOKE GLADSTONE:  Red font.

JONAH LEHRER:  Exactly. In the scientific literature the vast majority of citations, of course, are positive. They’re showing that they’re building on these earlier discoveries. This would be the opposite kind of citations.

BROOKE GLADSTONE:  Mm-hmm.

JONAH LEHRER:  The more ambitious, I think, fixes focus on things like forcing scientists to declare in advance what their hypotheses might be.

BROOKE GLADSTONE:  Mm!

JONAH LEHRER:  So, you know, in a lot of scientific fields, one of the problematic things has been that scientists, let’s say you’re doing some brain scan research and you're trying to find some significant correlation between people doing X and Y happening in some particular brain area, because of the standards of scientific publishing you’ll often have to get a result that passes the mystical test of significance, which shows that it’s very, very unlikely that these results were a byproduct of randomness.

BROOKE GLADSTONE:  Mm-hmm.

JONAH LEHRER:  So scientists will often keep on searching, keep on replicating the experiments until they get to that result that passes the test of significance. And so, the hope of having this kind of database where scientists have to register their original hypotheses is that it’ll make that harder to do, that scientists will have to say in advance what they’re looking for, and if they can’t find that original relationship, then that’ll be part of the paper they eventually publish.

BROOKE GLADSTONE:  It’s a kind of transparency.

JONAH LEHRER:  Exactly. I think the hope is that it would introduce a new kind of transparency to the scientific process. The larger point, of course, is that papers like this, you know, surge in retractions demonstrate and like this, you know, Ecosphere paper demonstrate, is that we can make the institutions of science a little bit better.

BROOKE GLADSTONE:  Jonah, thank you very much.

JONAH LEHRER:  Thank you.

BROOKE GLADSTONE:  Jonah Lehrer is a staff writer for the New Yorker.