To see the future of Cochrane threatened foreshadows a disaster for all of us

Sometimes it feels necessary to state the bleeding obvious—particularly when there’s blood on the floor. The tens of thousands of people around the world who create Cochrane and its summaries of evidence are contributing to one of humankind’s greatest scientific endeavors.

For 25 years researchers, clinicians, consumers, policy-makers and others have been using the rigorous new tools of an evidence-informed approach, to collectively produce systematic reviews about what works and what doesn’t in healthcare. Notwithstanding limitations and uncertainties, the Cochrane Review has rightly become something of a gold standard in evidence, and whenever a friend or family member has a question about their care—including questions of life or death—it’s the first place I send them.

What’s at stake in the current bloody fight unfolding within Cochrane’s Governing Board, is not just the credibility of individuals or organisations, it’s the future of reliable trustworthy evidence in a world of increasing falsity and fake news. [1,2] To see this future threatened foreshadows a disaster for all of us.

Unlike almost everything I’ve ever written, this piece is a desperate personal plea to the many clever and cool heads within the extended Cochrane family to try and calm this crisis, heal the rifts, and turn this challenge into an opportunity to enhance public trust, not squander it. No one asked me to write this opinion piece and writing it may mean I lose my role hosting The Recommended Dose podcast—funded by Cochrane Australia and co-published with The BMJ—but I’ve spent too long watching this tree grow to stand by and watch it burn to the ground.

When I first started reporting on healthcare almost 25 years ago, reading Archie Cochrane’s simple book Effectiveness and Efficiency was life changing. [3] The searing scrutiny of evidence-based medicine was starting to reveal that the benefits of many medical interventions were being routinely exaggerated, and their harms played down. Twenty years ago the television documentary that accompanied my first book, took me to Welsh coal mining districts where Archie cut his epidemiological teeth, to the tiny first Cochrane centre in Oxford, and to McMaster in Canada, where this radical new approach was incubating.

This piece is not about who is right in the current fight, but about reinforcing the rightness of the giant global collaborative project that is Cochrane. My personal views on the dispute over the HPV vaccine review are unimportant and beyond the scope of this opinion piece—suffice to say what appears to be some overstatement of criticism has precipitated what looks like an overreaction. [4] I would respectfully suggest that if there was a pill—or a course of cognitive behavior therapy—that meaningfully reduced the symptoms of hubris and promoted humility, I would recommend key players take a strong dose. Starting yesterday.  

More seriously, I would wish every success to those at the Cochrane Colloquium in Edinburgh this week who must be striving through late-night and early-morning meetings to try and heal the wounds as quickly as possible. Potential short-term solutions to the crisis will be complicated and of uncertain benefit—familiar challenges for those dealing with healthcare evidence.

What’s more certain is that the current challenge can serve as an opportunity. The much bigger crisis here is the threat to the reliability of healthcare evidence and public trust posed by the unhealthy financial entanglement between industry and those who evaluate and use its products. [5]

Fifteen years ago, the Cochrane Collaboration was at a crossroads in its relationship with pharmaceutical companies. [6] It opted then to tighten its policy and firmly reject the idea of companies sponsoring Cochrane reviews. Yet Cochrane policy, renewed again in 2014, still allows individuals with financial ties to pharmaceutical companies to review evidence about those same companies’ products—if they constitute a minority of the review team.

Given what we know about the systemic bias introduced into industry sponsored studies, the egregious nature of much of industry’s marketing behavior, including its work with key opinion leaders, it’s anathema that conflicted individuals should be reviewing what is often conflicted evidence to start with. Cochrane has an opportunity to provide global leadership by cleaning up this mess—as The BMJ is attempting to do with its new policy of seeking out non-conflicted researchers to author influential educational material. [7]

Related to this reform should be explicit new ways to address and investigate the under-investigation and under-reporting of harms in medical studies—already flagged by Cochrane leadership in recent correspondence about the current controversy. [8] And thirdly, as part of a drive to enhance public trust and get closer to the truth about which particular groups of people might benefit from medical interventions, Cochrane reviews might throw a lot more explicit scrutiny on the sometimes controversial definitions of disease on which they rely—highlighting the inappropriately lowered diagnostic thresholds which can drive overdiagnosis and overtreatment.

These words are offered respectfully, not as partisan criticisms to support one side or the other in the current dispute—but rather with a hope for calm, cool, conflict-resolution—from a long term observer with a strong interest in seeing this global collaboration survive and thrive. Like many of us, I want to see this tree continue to grow and bear fruit for a long time to come.  

Ray Moynihan, senior research fellow, Centre for Research in Evidence Based Practice, Bond University, Australia.

Conflict of interest statement: Ray Moynihan has a contract to present The Recommended Dose podcast, funded by Cochrane Australia. He is also a long-time contributor to BMJ.

References:

1. Marcus A, Oransky I. Turmoil erupts over expulsion of member from leading evidence-based medicine group. Stat News. September 16, 2018.
2. Koster M, Burton M. Message from the Governing Board. Cochrane (website) September 15, 2018
3. Cochrane A. Effectiveness and Efficiency: random reflections on health services. Cambridge: Cambridge University Press; 1972.
4. Hawkes N. Cochrane HPV vaccine review: BMJ journal defends “inconvenient criticisms” BMJ 2018;362:k3927.
5. Lo, B, Field M.(eds) Institute of Medicine (US) Committee on Conflict of Interest in Medical Research, Education, and Practice. Washington DC: National Academies Press; 2009
6. Moynihan R. Cochrane at crossroads over drug company sponsorship. BMJ 2003;327:924
7. Chew M, Brizzell C, Abbasi K, Godlee, F. Medical journals and industry ties. BMJ 2014;349:g7197.
8. Tovey D, Soares-Weiser K. Cochrane’s Editor in Chief responds to a BMJ Evidence-Based Medicine article criticizing the Cochrane Review of HPV vaccines. Cochrane (website) September 3, 2018.