Gathering evidence is clear straightforward science. Right? Not if you believe the speakers at “Evidence Live.” Today’s presentations were a challenge to believers. Fraud, misconduct, non disclosure of data, and conflict of interest were just part of the problem. Peter Wilmshurst (University Hospital of North Staffordshire) whose dedication to the truth in the face of lawyers and pressure from pharma and manufacturers, entranced us with his career long battles for the truth. He catalogued a series of attempts to alter, manipulate, and influence research evidence in both drug and technology development. Tom Jefferson (Cochrane Acute Respiratory Group), who continues to seek the data on Tamiflu, believes that 60% of evidence is not published, and often only part of the information is made available. The public have invested a huge amount of money—we have a right to know what a drug does.
Ben Goldacre (writer and broadcaster) is passionate about data disclosure. I liked his reasons: deliberate suppression of negative studies that inflate the possible efficacy of a product mean that, even if the drug is ineffective, patients are exposed to potential side effects. And, they may be deprived of a more effective treatment. We would consider it unacceptable if researchers withheld data points in a research paper—we should be equally incensed by non disclosure of negative studies. But, we don’t consider non publication as research misconduct.
Mike Kelly (Centre for Public Health, NICE) explained that our concepts of evidence based medicine are derived from physics—cause and effect. But that’s not the way it works in public health—so many factors interact between evidence and behaviour. Brian Haynes (ACP Journal Club) also addressed the gap between evidence and practice. Even with quality evidence supporting an intervention, implementation is limited both by physician uptake and patient adherence. Evidence is essential but not sufficient. A speaker from the floor felt there was a particular problem in primary care, but Brian pointed out that primary care doctors have the greatest challenge in absorbing and implementing evidence in dealing with so many conditions and such range of disease. He set us a target—to see quality evidence supporting 90% of clinician decisions by 2020—an ambitious objective.
Even systematic reviews attracted criticism. Jack Cuzick (Wolfson Institute of Preventive Medicine) is concerned about pressure to adhere to PRISMA. Its not a panacea and he feels there is too much focus on the checklist and not enough on the clinical issues. These are guidelines and should not be proscriptive—we need to use our judgement in including trials. Common sense matters: including very different trials in a meta-analysis may be completely inappropriate. And, I also agreed with his view that undertaking reviews should not be a career in its own right, but part of a subject matter activity.
Jack Wennberg (Dartmouth Institute for Health Policy and Clinical Practice) listed additional factors. He cited supply sensitive care as the explanation for the wide variation in clinical practice between Boston and New Haven, backed up by similar variation in hospital discharge data. Both are supported by world class universities and hospitals (Harvard and Yale), but there was a consistent difference in activity by a factor of 1.6. He also described preference sensitive care and how there was an interaction between evidence of effectiveness and patients’ preferences. There is more to clinical practice than evidence. Pointing out that we don’t really know the correct level of service provision, he described rationalisation in the US as chaos but, that in the UK as organised chaos.
Domhnall MacAuley is primary care editor, BMJ.