Neal Maskrey: Where is the evidence for evidence based medicine?

neil_maskreyThe internet was 25 years old last week. When Tim Berners-Lee was asked what has surprised him most about the internet’s use he replied “kittens.” With respect, I disagree. The most surprising aspect of the internet is how easy it is to genuinely interact with people around the world. My e-friends might not come to sit at my sick bed and in time share my “dark, tender moments,” but I have e-friends with whom, thanks to Sir Tim, I freely share sadness, joy, laughter, tears……and lots of ideas.

Last week I hope I made a new e-friend. She asked where was the evidence that using evidence to practice medicine is actually beneficial. “So many medical societies have got into the guideline business.” Does adherence to these guidelines really have a positive impact on healthcare?

What’s the evidence for evidence? Good question. There are indeed papers describing changes in management and reductions in mortality and morbidity with guidelines. Hoped for effects with new medicines don’t always materialise and evidence has prevented and corrected much inappropriate treatment.

But it’s immediately obvious that teasing out the effect of about 20 years of evidence based practice (EBP) from multiple other changes in population health and healthcare is difficult. And too often we find evidence looks great, but then discover it has been distorted, and our faith in people and systems suffers.

It’s probably more helpful to think of EBP as a social movement. “Evidence” is now culturally part of healthcare in many parts of the world—in the UK I think we often underestimate that change. You don’t see people getting antibiotics, paracetamol, and a decongestant for a cold anymore because we unconsciously follow the evidence that they don’t do any good. We don’t measure that change and demonstrate that EBP “works.” In skilled clinicians’ hands, EBP has imperceptibly become an integral and positive part of providing care.

My other point about social movements is that there’s a moral aspect. What would you like clinicians to use in advising about the best management options? The best available scientific evidence or something else? It would be equally hard to find “evidence” that universal suffrage, freedom of the press, abolishing slavery, or teaching medical students anatomy “works” of course.

Then there’s the problem that human beings don’t make decisions by blindly following what other people tell them. To say nothing of the impossible task of keeping up with the volume of published evidence. So new e-friend, looking for guidelines being followed isn’t the right question. The cultural influence of EBP has to be greater than a simple, linear guideline effect.

There are richer pickings in questioning the variable quality of many guidelines. When only 5% have a described search strategy, grade the evidence, and report the type of stakeholders (not exactly high hurdles), much of this is poor quality noise, not signals. EBP Rule #1: know how the guideline was produced.

Evidence based practice and the internet. Both young. Both flawed. But we can’t seriously imagine life without either of them.

Competing interests: I declare that that I have read and understood the BMJ Group policy on declaration of interests and I hereby declare the following interests: Employed part time by the National Institute of Health and Care Excellence.

Neal Maskrey‘s early career was as a GP before spending seven years as a medical manager and part time GP. After twelve years as a director of the National Prescribing Centre and programme director at NICE, he is now honorary professor of evidence-informed decision making at Keele University, and consultant clinical adviser in the Medicines and Prescribing Centre, NICE.

  • MD_uk

    The question sometimes is, what if the evidence is incomplete, and available treatments are not very effective? Evidence says cognitive behavioural therapy is the best available option for chronic low back pain, and physical therapy can be effective for acute, but not chronic pain. My doctors believed that, I believed that. Indeed, a couple of physical therapists failed to help, and then faithfully followed my CBT pain management program until I deteriorated into serious disability. A friend convinced me to try out another therapist, particularly skilled working with intractable pain. She promptly came up with a hypothesis about the source of trouble, treated it, and helped me to come back to near normal functioning.

    I struggled throughout this process, with making decisions “against evidence”, and with trying additional modalities like acupuncture on my PTs advice, when I didn’t believe they were truly supported by evidence – and yet they did help. In retrospect, I wished that the interpretation of current evidence was “you have to try a lot of things until you find what works, but nothing is very effective” rather than “CBT is the most effective method, and painkillers and ongoing physical therapy don’t really work, so it’s not worth trying them”.

  • EMDS

    As Dr. Maskrey will be well aware, another problem that besets EBP is that “evidence” is in the eye of the beholder.

    Just look at the controversies over the value of screening for risk of breast cancer and prostate cancer. Differing groups of “experts” operating from differing perspectives and differing points of view can have radically different interpretations of precisely the same data sets!

  • Nick Timlin

    Hi Neil attended a great talk by yourself in the Gateshead Hilton last year on decision making.As you said then without invoving patients in the decision making and looking at compliance issues the outcomes promised are much reduced