You don't need to be signed in to read BMJ Blogs, but you can register here to receive updates about other BMJ products and services via our site.

The way forward from ‘rubbish’ to ‘real’ EBM in the wake of Evidence Live 2015

24 Apr, 15 | by BMJ

by Huw LlewelynHuw Llewelyn

At Evidence Live, Iona Heath (video) reminded us that EBM should not interfere with wisdom and common sense. Trish Greenhalgh (video) gave a hilarious example of ‘rubbish EBM’ when a ‘falls protocols for elders’ was invoked after she went over her bicycle handlebars at speed! The answer is to arrive at initial impressions and decisions based on one’s current knowledge and then AFTERWARDS to check them against transparent reasoning and evidence. If they do not concur, then the original impression should be reconsidered or the evidence-based recommendation questioned (but this needs to be done in full knowledge and understanding of its evidence base).

The step that connects intuitive commonsense to transparency and evidence is REFLECTION as to which diagnosis led to each decision and which symptoms, signs and test results (i.e. the ‘patient’s particular evidence’) led to both. We have all had to do this in exams and we still do so when referring patients to someone else or replying in letters or discharge summaries. In other words, we have to provide a TRANSPARENT rationale that supports our ‘intuition’ by linking each decision to the relevant symptoms, etc., usually via a diagnosis.
One idea might be to create libraries of concise example justifications, cross referenced to fuller evidence-based information like that in BMJ Clinical Evidence. These concise guidelines could be used to check quickly whether an intuitive impression can be justified reasonably. The guideline chosen to justify could be also be copied electronically, edited and given to the patient. This would enhance shared decision making. It could also include other active problems to form a transparent evidence-based clinical summary to improve communication.
A transparent DIAGNOSTIC rationale is created by showing that all but one (ideally) of a list of differential diagnoses are improbable. This happens if other findings are common in some of the diagnoses and rare in others (i.e. giving strong SENSITIVITY ratios). The Oxford Handbook of Clinical Diagnosis contains about 550 pages of brief guidelines for justifying such diagnostic reasoning. EBM has hitherto focused instead on LIKELIHOOD ratios, which are difficult to replicate. This is mainly because ‘1-specificity’ can vary greatly between study populations depending on their size and the presence or absence of other diagnoses. My presentation[SessionE4,abstract#5] showed that despite some findings with individual likelihood ratios of 1 being regarded as useless, when their ‘sensitivity ratios’ were used instead with the ‘reasoning by elimination theorem’, they predicted a diagnosis correctly with certainty.
Another problem is that little work has been done on evidence-based treatment and diagnostic criteria. It is only with such appropriate criteria that it will be possible to assess and optimise the ability of findings to predict diagnoses and treatment benefits. Jon Deeks described ‘umpire’ tests that suggest which findings might form ‘better’ diagnostic criteria. For treatment, ‘umpire’ findings would have to be outcome benefits based on RCT data and ideally those outcomes that affect patients directly. Without such evidence, there always will be avoidable over-diagnosis and over-treatment, and also failure to advance ‘stratified medicine’.
So, to move forward, we also need common sense supported by ‘transparency guidelines’ that are founded on evidence-based diagnostic criteria, ideally derived from RCTs, and probabilities of the differential diagnoses of symptoms, signs, test results and their relevant sensitivity ratios.

Huw Llewelyn is the originator of the Oxford Handbook of Clinical Diagnosis, based on the teaching methods he developed as a consultant physician at King’s College Hospital London and also based on his research into the set theory of diagnosis.  He has 40 years experience of diagnosis and management in acute settings, wards and clinics (and in general practice as a registrar).  He has also been an honorary fellow in mathematics at Aberystwyth University for 20 years.


(Visited 98 times, 1 visits today)
By submitting your comment you agree to adhere to these terms and conditions

Leave a Reply

You can follow any responses to this entry through the RSS 2.0 feed.
BMJ Clinical Evidence Blog homepage

BMJ Clinical Evidence

Clinical Evidence is a database of systematic overviews on the effectiveness of key interventions, together with tools and resources to learn and practise EBM. Visit site

Creative Comms logo

BMJ Clinical Evidence latest news

BMJ Clinical Evidence latest news