Christine Stirling: Move over RCT—time for a revised approach to evidence based medicine

We are stifling our capacity to use research evidence for patient centred outcomes by privileging RCTs

In the two decades since Sackett et al outlined their definitive account of evidence based medicine (EBM) and evidence hierarchies in The BMJ, randomised controlled trials (RCTs) have gained a position of overwhelming dominance that appears to surpass the original intent of EBM.

From the first publication of an RCT in 1948 to today, RCTs have come to dominate the evidence landscape, gaining both research prestige (which translates to funding and publications) and medical authority. RCTs are now often considered the best form of evidence in most evidence hierarchies—particularly if combined as systematic reviews. I find myself increasingly frustrated with this status when I participate in grant review panels and believe it is time to shake things up and speed up changes to this mindset.

There has already been increasing acknowledgement that the EBM movement should reconsider RCTs’ pre-eminence and be open to evidence collected from other methods. The limitations of RCTs are becoming more widely recognised. Some argue that RCTs are reductionist, and that they fail to incorporate patient values and complexities. It’s been pointed out that even an ideal RCT has limited generalisability in answering research questions that help us understand which treatments work for which individuals. And it’s been highlighted that, despite commentators’ praise for how RCTs prevent bias and confounding, in reality many of them have enough flaws that they cannot justify quality claims over observational studies.

Research evidence users (including health professionals) can still be led to frequently ignore these “lower levels” of evidence, with the number of Cochrane reviews that only consider RCTs a good indication of this. We are starting to see some adjustments to hierarchies in new guidelines, such as GRADES, although they still privilege RCTs and can inappropriately ignore other valid forms of evidence. NICE guidelines are another example of efforts to include different levels of evidence and expert opinion into clinical recommendations. Yet RCTs are still well entrenched into our research training, funding, and publication systems as “gold standard’.

While it will take many years to unravel the systems and infrastructure that prioritise RCTs, health professionals and educators can still easily move towards using a wider range of evidence. This will require firstly a change in mindset and adjustments to education curricula, but guidance is available for thinking about the best study design to use in relation to the clinical question.

If we wish to improve clinical care, health services, and patient outcomes we will need to continue developing new ways of valuing and using evidence. Person centred medicine provides an impetus to do this, with its focus on models of clinical decision making that balance patient values and individual traits with clinician expertise and available knowledge.

Providing care in the era of “big data” also aids this endeavour. With a larger volume and variety of data to access and analyse, observational studies may deliver better quality evidence, which incorporates subgroup analysis and heterogeneity more cost effectively than RCTs. Linking N of 1 studies through large databases could also strengthen this individual trial method and provide impetus for more clinician based research. The time is ripe for further developing this precision research approach, with increasingly technology aware patients creating the potential for coordinated N of 1 trials that research with patients. An Australian example of this resulted in treatments changes for 28 of 64 children with attention-deficit/hyperactivity disorder.

For many clinicians the privileging of RCT evidence over other valid forms of evidence impacts clinical decision making as it leaves them with a limited range of evidence to use. As our capacity to share and process information globally increases, so does our ability to analyse large data sets, but this doesn’t have to take the form of data in the traditional sense and it doesn’t have to be only clinicians and researchers who are doing the sharing. Patient blogs and forums allow patients to share stories globally and for anecdotal treatment results to be publicly documented—providing a rich source of patient stories and outcomes.

Our ability to communicate and analyse big data needs to be met with revised methodologies and innovation, but this requires a wider debate, changes to funding priorities, and a focus on evidence that fits the clinical question. With revised research evidence frameworks and methods, health professionals could have improved access to, and understanding of, more relevant research evidence for clinical decisions.

Christine Stirling is a mixed methods health services researcher with the University of Tasmania.

Competing interests: None declared

  • Eunice Minford

    Thank goodness for some sense on EBM at last. Totally agree RCTs over rated and not the only valid form of evidence. Certain factions trying to control who gets to say what is valid and what not. Patient experience and outcomes essential and very valid.

  • Dr Kevin Doyle

    One size fits all, when it clearly does not, and yes controlled alright, largely by vested interests.
    If this is what we consider as gold standard, gold has lost it shine.

  • Sebastián González-Dambrauskas

    Nice viewpoint with an uncommon common sense. Lab conditions like those used in RCTs cannot answer real life questions as we know but EBM movement has failed in moving forward. Most RCTs are corrupted from COI if diverse origin and we as societies cannot afford that more than 85% of clinical research is wasted (recall I Chalmers and GLasziou Lancet 2009). To put things worse, EBM has been hijacked as JP Ioannidis states. Healthcare value equation states that Value= Outcomes/costs. Within outcomes, we must include always the patients and their families first without forgetting social costs. In my view we must move forward value-based medicine and the only path for that is through independent clinical research and looking at the real problems of our societies. To detect which research is valuable and which are not. Only global and independent collaborative iniciatives and networks can make that duty. Never is late for a change!

  • Sebastián González-Dambrauskas

    Nice viewpoint with an uncommon common sense. Lab conditions like those used in RCTs cannot answer real life questions as we know but EBM movement has failed in moving forward. Most RCTs are corrupted from COI if diverse origin and we as societies cannot afford that more than 85% of clinical research is wasted (recall I Chalmers and GLasziou Lancet 2009). To put things worse, EBM has been hijacked as JP Ioannidis states. Healthcare value equation states that Value= Outcomes/costs. Within outcomes, we must include always the patients and their families first without forgetting social costs. In my view we must move forward value-based medicine and the only path for that is trough independent clinical research and looking at the real problems of our societies. To detect which research is valuable and which are not. Only global and independent collaborative iniciatives and networks can make that duty. Never is late for a change!

  • Chris Del Mar

    Thanks for citing an n-of-1 trial I co-authored. But Christine Stirling doesn’t seem to realise that n-of-1 trials are actually a form of RCT! They were nominated by Gordon Guyatt (one of the doyens of EBM that Christine decries) as providing more information than even RCTs for individual patient treatment questions.
    Guyatt, G. H., A. Heyting, R. Jaeschke, J. Keller, J. D. Adachi, and R. S. Roberts. 1990. ‘N of 1 randomized trials for investigating new drugs’, Controlled Clinical Trials, 11: 88-100
    However Christine makes a common error when she decides when to use different study types such as RCTs, observational studies (which include Big Data), and the rest. The best evidence for treatment questions comes from randomised trials (including n-of-1!). Observational studies risk confounding issues that can lead us in the wrong decision. Look at the mess doctors got into by using observational studies to decide that postmenopausal women would benefit from HRT to prevent CVD. A later RCT proved us completely (ie 180 degrees) wrong. (JAMA 2004, 291: 1701-12)

    People who want aa full description of why and wherefore of the need for RCTs should read this (for free)

    Evans, I, H Thornton, I. Chalmers, and P. Glasziou Paul. 2011. Testing treatments. Better research for better healthcare [available for free at http://www.jameslindlibrary.org/testing-treatments.html%5D (Pinter & Martin Ltd: London, UK).

    Chris Del Mar

  • Going back to anecdotal evidence would certainly be welcomed by the pharmaceutical industry, and still more welcomed by the legions of quacks, But it would harm patients and lead to squandering of money on ineffective treatments.

    I suspect that one reason for this sort of hubris is that there are so many conditions for which effective treatments don’t exist. That leads to clutching at straws provided by unreliable data. For example, it’s a great pity that there is such a lack of effective treatments for non-specific low back pain. But it does no favours to patients to pretend that good treatments exist.

  • Andy Lewis

    Dr Minford. I take a quick look at your website and I see you offer “Sacred Esoteric Healing is a gentle form of hands on healing on the clothed body that is deeply relaxing and helps people to reconnect back to their essence.”

    What valid evidence do you use to understand the effectiveness of esoteric healing practices?

  • Christine Stirling

    Thank you for your feedback Chris Del Mar. Indeed N of 1 trials can be run as a form of randomised control trial, but not necessarily. Further the example you provide is actually a great example of the damage that can come from failure to correctly acknowledge the limitations of RCTs. Millions of women taken off HRT, and still to this day some with significant symptoms find it difficult to access HRT. For more up to date understanding of the HRT evidence people could look at:
    R.J Baber, N. Panay & A Fenton the IMS Writing Group (2016) 2016 IMS Recommendations on women’s midlife health and menopause hormone therapy. Climacteric, 19:2, 109-150
    Or
    NICE guideline [NG23] Menopause: diagnosis and management (2015) available at http://www.nice.org.uk/guidance/ng23

  • Christine Stirling

    Thank you for your comments. I agree we should spend more time considering the value of particular research approaches. RCTs have a place in research but can force people to ask narrow questions that fit the method.

  • Jim Vine

    RCTs conducted with highly controlled lab-like conditions are problematic if the objective is to understand how treatments will work in real practice. But that is an argument for conducting RCTs with a more ‘pragmatic’ attitude, in the sense defined by Schwartz and Lellouch (1967). https://doi.org/10.1016/j.jclinepi.2009.01.012

    Table 1 in the 2008 CONSORT pragmatic trial extension is a good summary of approach. https://doi.org/10.1136/bmj.a2390

  • Jim Vine

    Frank Harrell and Laura Lazzeroni have blogged about an example where an RCT with 64 patients is as accurate as an infinitely large observational study: http://www.fharrell.com/2017/06/ehrs-and-rcts-outcome-prediction-vs.html