{"id":41367,"date":"2018-02-09T15:13:04","date_gmt":"2018-02-09T14:13:04","guid":{"rendered":"https:\/\/blogs.bmj.com\/bmj\/?p=41367"},"modified":"2018-02-16T17:49:33","modified_gmt":"2018-02-16T16:49:33","slug":"jeffrey-aronson-when-i-use-a-word-the-wheel-of-evidence","status":"publish","type":"post","link":"https:\/\/blogs.bmj.com\/bmj\/2018\/02\/09\/jeffrey-aronson-when-i-use-a-word-the-wheel-of-evidence\/","title":{"rendered":"Jeffrey Aronson: When I Use a Word . . . The wheel of evidence"},"content":{"rendered":"

\"jeffrey_aronson\"<\/a>Last week I discussed the concept of <\/span>teleoanalysis<\/span><\/a>, in which different types of evidence from disparate sources are analysed either simultaneously or sequentially. To be clear, the term implies not the very last analysis that could be done, but a thorough systematic analysis of all the evidence available at the time, from, for example, randomised trials, non-randomised studies, epidemiological studies, n-of-1 studies, case reports, and any other forms of adducible evidence, such as laboratory evidence and <\/span>mechanistic arguments<\/span><\/a>. Teleoanalysis can be a quantitative analysis of quantitative data or, if the data do not lend themselves to formal analysis, a narrative account. Furthermore, the results of qualitative studies can be used as an adjunct to a quantitative teleoanalysis or as part of a narrative teleoanalysis, aiding interpretation. If other evidence becomes available, it can be included in a revised teleoanalysis.<\/span><\/p>\n

This concept reminds us that it is fallacious to think of evidence as being arrayed in hierarchies, based on the idea that some forms of evidence are intrinsically more reliable than others. The first <\/span>hierarchies<\/span><\/a> put systematic reviews of randomised controlled trials at the top (high quality) and anecdotal reports and expert opinion (wrongly categorised\u2014opinion is not evidence) at the bottom (low quality). However, it has gradually been recognised that different types of evidence have different uses and that in some circumstances the varieties in the middle of the list or at the bottom may have as much value as those at the top. Hierarchies distort this.<\/span><\/p>\n

Randomised studies, it is generally agreed, provide the most convincing quantitative evidence. Controlled n-of-1 studies, in which the effects of an intervention are studied in an individual and compared with a placebo or other comparator, can give excellent evidence in the individual. However, such designs have limitations. For example, the results of a randomised study may not be applicable to the general population (the problem of <\/span>external validity<\/span><\/a>) and are not good at detecting rare events; n-of-1 studies are applicable only in some conditions.<\/span><\/p>\n

Observational studies are often much larger than randomised studies and although confounding is a major problem they can give information that randomised studies cannot give or not as readily, <\/span>particularly<\/span><\/a> in poorly resourced settings. Meta-analysis of randomised studies can elucidate <\/span>dose response relations<\/span><\/a> when they have involved a range of doses, but if not observational studies can do the job instead. If a drug-event pair has been shown to be causatively linked in a randomised study, the risk estimate of the association derived from meta-analyses of observational studies is <\/span>similar<\/span><\/a> to the risk estimate calculated from a meta-analysis of randomised studies. In studying drug-drug interactions it may be impractical to study enough individuals to reveal an effect, especially if the interaction affects only a subset of individuals. For example, a large observational <\/span>study<\/span><\/a> showed that paracetamol potentiates the anticoagulant action of warfarin, and the more paracetamol you take for longer, the bigger the effect is; this dose and time related adverse drug-drug interaction was first demonstrated in a case control study of patients with an INR of 6 or over and was not seen in small randomised studies. Similarly, in a case-control <\/span>study<\/span><\/a> of the interaction of laxatives with acenocoumarol or phenprocoumon, lactulose increased the risk of an INR over 6 by 3.4 times.<\/span><\/p>\n

Sometimes even an anecdotal report can elucidate a beneficial or harmful effect without the need for a randomised study. A whimsical <\/span>example<\/span><\/a> of this is the anecdotal evidence, in the absence of randomised studies, that parachutes \u201c[prevent] death and major trauma related to gravitational challenge\u201d. Other examples show that anecdotes can provide convincing evidence of both <\/span>benefits<\/span><\/a> and <\/span>harms<\/span><\/a> without the need for randomised or observational studies.<\/span><\/p>\n

Similar comments could be made about other types of evidence, mechanism based, laboratory based, and qualitative evidence, each of which has a place in the evidence spectrum.<\/span><\/p>\n

The word \u201cteleoanalysis\u201d comes from the Greek adjective \u03c4\u03ad\u03bb\u03b5\u03bf\u03c2, itself derived from an IndoEuropean root K<\/span>W<\/span>EL, which implied different forms of turning, as I <\/span>discussed<\/span><\/a> last week. This suggests depicting different forms of evidence and their interrelations as a wheel, with no predominant class. Recognising that all forms of evidence can be of value in different ways emphasises the fallacy of hierarchies and suggests a different way of thinking about how evidence should be analysed. This is illustrated and described in the figure below.<\/span><\/p>\n

\"\"<\/p>\n

Teleoanalysis and the wheel of evidence.<\/span><\/p>\n

The green ring shows the different classes of evidence and the blue ring shows some of the types of studies that they comprise:<\/span><\/p>\n