In an address to oysters, the Walrus in Lewis Carroll’s “The Walrus and the Carpenter” proclaims:
The time has come, [the Walrus said],
To talk of many things:
Of shoes—and ships—and sealing-wax —
Of cabbages—and kings—
And why the sea is boiling hot—
And whether pigs have wings.
I’m not sure where the Walrus was going with this but what is clear is that wherever he was headed, action was needed on a broad front. Discussing cabbages alone simply would not do.
The same is true of evidence-based medicine (EBM). While it is hard to argue with the basic fact that all healthcare decisions would do well to be informed by some research evidence, there are growing rumblings of discontent about EBM, and sometimes hostility. Why?
I’m not a health professional, I’m a trial methodologist. I don’t have to make daily use of evidence for healthcare decisions, except my own and those of anyone else who invites me into their decision. But it seems to me that the world of healthcare evidence is both overwhelming and underwhelming. Anyone who cares to look will find a lot of evidence out there, some of it packaged into systematic reviews or guidelines, some of it standing alone hoping someone will notice. This is the overwhelming part: if I’m travelling hand-luggage only I pause before printing out most Cochrane systematic reviews. Some guidelines need GPS to navigate them.
The underwhelming part comes when you take a closer look, especially when you look at the trials that form the heart of much of the evidence we hope will inform treatment decisions. To cut to the chase, many trials are poorly informed by earlier research, poorly designed, badly executed, reported opaquely and of little relevance to the people whose decisions they were supposed to be supporting. In other words they are rubbish. I helped write a summary a year or so ago for a systematic review that asked a question about which of two types of catheters was least likely to lead to infection [1]. After looking at 42 trials involving over 4500 people the reviewers concluded that they hadn’t a clue which is best. A wonderful but sad piece of work by Céline Habre and colleagues updated another systematic review and concluded that two-thirds of the 136 trials done over the 10-year period since the previous version were clinically irrelevant. [2] Two-thirds, wow.
Research evidence should have a role in healthcare decisions; it will rarely be the only thing to consider and it might be ignored. The point is that it should be on the table. No serious person wants evidence-free healthcare. But the role of evidence in healthcare needs help.
Like the Walrus, the authors of the EBM Manifesto 2.0 suggest work is needed on many fronts. They want work on greater roles for patients in research design, better research methods, better reporting, more informative systematic reviews, better use of evidence in policy, reductions in unnecessary medicalisation and improvements in the ability of health professionals to recognise poor quality evidence to mention but some. I like it.
But for my part, the area we really need to work on is how we do research: we should do more research on research. In my world of trials, it is astonishing how little attention is often paid to existing research when designing a trial, how little evidence we have to support our own decisions about design, conduct and reporting and how little thought we sometimes give to what the intended users of our trials actually need from them.
Manifestos should be a spur to action, although what they lead to may not always be what was envisaged when they were written, and manifesto promises do have a habit of quietly disappearing. Sometimes it’s hard to deal with sealing-wax and and shoes when there are also boiling seas to deal with. But if I was forced to choose my king over a cabbage, I’d choose to take aim at improving the way we design research, especially trials. The Walrus would be pleased I’m sure, whatever he was talking about.
Shaun Treweek is a health services researcher at the Health Services Research Unit at the University of Aberdeen. He is working on ways in which we can improve the efficiency of randomised trials and he leads the Trial Forge initiative.
I have no competing interests.
Follow Shaun on twitter: @shauntreweek
References:
- Kidd EA, Stewart F, Kassis NC, Hom E, Omar MI. Urethral (indwelling or intermittent) or suprapubic routes for short- term catheterisation in hospitalised adults. Cochrane Database of Systematic Reviews 2015, Issue 12. Art. No.: CD004203. DOI: 10.1002/14651858.CD004203.pub3.
- Habre C, Tramer MR, Popping DM, Elia N. Ability of a meta-analysis to prevent redundant research: systematic review of studies on pain from propofol injection. BMJ 2014;349: g5219–9.