On January 14th I published a critique of an editorial by Thomas Sherman and Adriane Fugh-Berman on the Hasting Center for Bioethics blog. A few days later they were moved to respond.
The issue at hand was the retraction of the notorious Séralini rat study.
Sherman and Fugh-Berman held that the retraction was the result of industry pressure and that the retraction didn’t cite reasons that fell within accepted guidelines for retraction [pdf]. I quote them at length to avoid misrepresenting them.
According to the Committee on Publication Ethics, a group that advises medical editors and publishers on ethical issues, particularly, how to handle cases of research and publication misconduct:
Journal editors should consider retracting a publication if:
- they have clear evidence that the findings are unreliable, either as a result of misconduct (e.g. data fabrication) or honest error (e.g. miscalculation or experimental error)
- the findings have previously been published elsewhere without proper crossreferencing, permission or justification (i.e. cases of redundant publication)
- it constitutes plagiarism
- it reports unethical research
There are hundreds of studies that should be permanently removed from the scientific literature, but the Séralini study is not one of them. The FCT retraction announcement very clearly states: “Unequivocally, the Editor-in-Chief found no evidence of fraud or intentional misrepresentation of the data” – and then goes on to say, incredibly, that the study is being withdrawn because the journal’s own review of the primary data show that the results are inconclusive.
Inconclusive? Until a hypothesis is proven, all results are inconclusive.
It would have been perfectly appropriate for the journal to have written an editorial expressing its concerns. Instead, it seems the editors may have succumbed to industry pressure to do the wrong thing.
. . . The retraction of the Séralini study is a black mark on medical publishing, a blow to science, and a win for corporate bullies.
My response was three fold. I agreed that the retraction had a political element, but that it did not seem to be in response to industry pressure.
Second, Sherman and Fugh-Berman had ignored Séralini’s own ethical lapses. The two that I pointed out were his unusual and manipulative press embargo on the study and his decision to allow the rats to die from massive tumors rather than euthanize them. I did not bring up the conflict of interest that the funding of the study represented. This was a conflict of interest not stated in the paper. Séralini wrote in his book that he funneled industry money through CERES to obscure the funding sources for this study. He failed to disclose any conflicts of interest in the paper. That seems like a major no-no to me.
Third, Sherman and Fugh-Berman had thrown around a lot of innuendo about conflicts of interest. While conflicts of interest raise red flags and call for heightened scrutiny, they do not justify jumping to conclusions. Instead, they should be seen as presenting a hypothesis which should be tested. Sherman and Fugh-Berman say, “The quality of Séralini’s work aside, the process by which his paper was retracted reeks of industry pressure.” But how can you judge whether the retraction can be confidently attributed to industry pressure if you put the quality of Séralini’s work aside?
Writing for the Hasting Center’s Bioethic Forum blog, Adriane Fugh-Berman and Thomas G. Sherman ask some tough questions regarding the Séralini retraction.
According to SpinWatch, a European muckraking organization, 11 of the authors of letters to the editor slamming Séralini’s study had undisclosed financial relationships with Monsanto. In 2013, Paul Christou, the editor of Transgenic Research, coauthored an attack on Séralini and the FCT editors in his own journal, calling for a retraction of the study. Christou did not disclose his multiple conflicts of interest, including being an inventor on patents on GM crop technology, many of which Monsanto owns. Meanwhile, back at Food and Chemical Toxicology, a new position for an associate editor was filled by Richard E. Goodman, a University of Nebraska professor who previously worked for Monsanto, and who has a longstanding association with the industry-funded International Life Sciences Institute (ILSI). Months later, Elsevier, FCT’s publisher, announced the retraction.
The quality of Séralini’s work aside, the process by which his paper was retracted reeks of industry pressure. The progression of science is not the least bit linear, but the process has to proceed unencumbered by censorship of unpopular or commercially disadvantageous results. The peer review process is imperfect – there are countless bad studies in the medical literature – but peer review works best when the efforts of reviewers and editors are devoid of conflicts of interest and outside pressures. The self-correcting nature of science can only work when industry does not taint the process.
. . . It would have been perfectly appropriate for the journal to have written an editorial expressing its concerns. Instead, it seems the editors may have succumbed to industry pressure to do the wrong thing. The media coverage in the U.S. has been one-sided; criticism of Séralini’s study has been widely covered in mainstream press, while information about the conflicts of interest of critics have remained in the alternative press.
The article raises a number of issues worth addressing.
First, Fugh-Berman and Sherman fail to put the retraction in the context of Séralini’s own ethical lapses. There were lapses in both the execution of the study and in his handling of the publicity following publication.
Letting those tumors grow to the sizes they did was a major ethical lapse. In the context of the issues raised by the article though, what’s more germane is Séralini’s decision to embargo the release of the study. This was clearly done to foil critical coverage of a clearly weak study. Timing the release the study in tandem with his book tour was also highly questionable behavior for a scientist.
The evidence that the study presented was inconclusive and yet Séralini made confident conclusions. That was highly problematic. It’s one thing to publish inconclusive results. It’s another thing to portray the evidence as demonstrating something that it does not. Even more problematic is that he went around the world trumpeting his conclusions. “Data” from inconclusive studies shouldn’t end up plastered on picket signs. In the face of the avalanche of criticism and debunking his research received, the ethical thing for Séralini to have done would have been to withdraw the paper rather than promote it. The paper made have slipped through the initial peer review, but it was absolutely eviscerated in post publication peer review. That much is undeniable.
Regarding the insinuations the journal bowed to industry pressure. The incentives don’t really seem to point in that direction. For the industry, the retraction is a formality. The paper had already been universally discredited. It could only reflect poorly on the industry and stir up paranoia in those rallying to Séralini’s cause. How could the industry not anticipate just these sorts of articles, making just these sorts of insinuations?
The party that has the most to gain is the journal, Food and Chemical Toxicology. When publishing papers, the authors and the journals enter into a reciprocal relationship. The journals and the authors confer credibility and prestige upon each other. In this case, Séralini took the credibility and prestige that the FCT conferred upon him. He returned the favor by bringing down heaps of scorn and recrimination upon the journal. It’s hard not to imagine that the quantity and quality of submissions suffered. It’s hard not to imagine that the journal felt it needed to do something to rectify the situation.
I agree with critics that the retraction was political. I disagree that it was meant to harm Séralini’s reputation. He had managed that quite well all by himself. The politics behind the retraction almost certainly had to do with salvaging the journal’s reputation. Absent Séralini withdrawing the paper, it’s hard to blame them.
The quality of Séralini’s work aside, the process by which his paper was retracted reeks of industry pressure.
This is absolutely wrong. Questions about the motivation of Séralini’s critics are only valid if their criticisms are invalid.
This is what is infuriating about Fugh-Berman and Sherman’s piece. They state that the quality of the Séralini’s work is beside the point. This is wrong. They seem to think that the Séralini Affair is a he said/she said affair; as if it were impossible for bystanders to assess whose position is stronger. It isn’t. Anyone with an 8th grade science education can understand the issues with the paper. Unless they are trying not to. The insinuation that the motivations of those who slammed the study could be explained by conflicts of interest is beside the point. It is beside the point because Séralini’s work was clearly substandard.
It’s valid to be aware of conflicts of interest. It is a reason for heightened scrutiny. However, those potential conflicts only become salient when presented with questions which can’t be explained otherwise. We ask first order questions. Is the evidence and analysis consistent with basic principles of how we understand the world? Is the analysis solid? Do results seem consistent with common experience?
If those first order questions haven’t raised any flags, there is no point in asking a second order question about conflicts of interest. If the criticisms of the Séralini paper were unsound, then you should ask, What is driving this? There are cases that call for following the money. This wasn’t one of them. You don’t need to follow the money to understand criticism of poor quality science.
When you start hollering ‘Conflict of Interest’ before evaluating the evidence and analysis, it becomes a ‘Get of Jail Free Card’. It becomes an excuse for discounting inconvenient evidence. Asking about conflicts of interest should be safeguard against getting snookered. Instead, it becomes a way to justify motivated reasoning. Awareness of conflict of interest should be a tool for explaining weak evidence and poor analysis. Instead it becomes an excuse for dismissing strong evidence and sound analysis. It leaves you lost in a hall of mirrors, surrounded by industry funded research, revolving door regulators, and defending bad research that confirms your biases. It leaves you lost in a fever swamp of paranoia without firm footing.
Examining the soundness of the evidence and the strength of the analysis must come first. Then you can decide whether questions of funding and loyalties are relevant. This is how you maintain a firm footing and hew to solid ground. This is how you can use awareness of conflicts of interest to avoid motivated reasoning. Otherwise you are only fueling the fire of your own biases. Fugh-Berman and Sherman level charges of conflict of interest while dismissing the questions about the quality of Séralini’s work. This is upside down and backwards. They should know better.
1. ROUNDING UP SCIENTIFIC JOURNALS
Adriane Fugh-Berman and Thomas G. Sherman | Bioethic Forum | The Hastings Center | 10 January 2014
2. STUDY LINKING GM MAIZE TO TUMORS IS RETRACTED
Barbara Casassus | Nature | 28 November 2013
3. A RANCID CORRUPT WAY TO REPORT ABOUT SCIENCE
Deborah Blum | Tracker | Knight Science Journalism at MIT | 22 September 2012
4. THE SERALINI RULE
Skeptico | 18 June 2013
5. HOW CAN WE TELL WHAT ‘GOOD’ SCIENCE REALLY SAYS?
Terry Daynard | Terry Daynard’s Blog | 14 January 2014
6. EXPERT REACTION TO GM MAIZE AND TUMOURS IN RATS
Science Media Centre | 19 September 2012