You don't need to be signed in to read BMJ Blogs, but you can register here to receive updates about other BMJ products and services via our site.

How much health evidence is there in the world?

2 Apr, 15 | by BMJ Clinical Evidence

by Gabriel Rada

Picture Gabriel Rada
Thirty five years ago, Archie Cochrane reproached the medical profession for not having managed to organise a “critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials”.
Systematic reviews were invented to deal with this problem, thus enabling what we call evidence based medicine, and their number has being steadily growing.
In theory, having one high-quality, up-to-date, clearly summarised systematic review for each relevant question would suffice to fulfill Cochrane’s challenge. However, even though the production of systematic reviews was estimated five years ago at eleven per day (at least 3 times that number during 2014), the target is still far from being reached, and probably moving away.

Only a minimal portion of trials have been assessed in systematic reviews, even though tens of thousands of reviews have been conducted, and the situation is much worse if we look beyond randomised trials.

On the other hand, there are many systematic reviews addressing the same topic, and they get quickly outdated. Nowadays, it is almost impossible to know if a particular review represents the best available evidence, unless compared to other similar reviews or by conducting an exhaustive review of the literature. Guidelines and overviews of reviews have emerged as possible solutions, but they are even more difficult to keep updated, among many other problems.
However, our chances of making Archie Cochrane’s proposal a reality might be improving!
Epistemonikos project is a not-for-profit initiative, established six years ago with the aim of gathering all what is usually considered health evidence useful for decision-making (systematic reviews, studies, overviews, guidelines, structured summaries, etc), and to organise it by health question (also by specialty and subspecialty among other options). The project combines cutting edge software with the human effort of more than 500 collaborators, and its growth has exceeded even the most optimistic expectations.
Containing more than 50,000 systematic reviews, Epistemonikos is now the largest database of this kind for health decision-making. The predefined strategy to identify them all, which includes screening more than 30 databases, will be completed during this year (the final number up to 2015 should be in the range of 80,000 to 120,000).
Additionally, Epistemonikos not only indexes reviews but also all studies included in those reviews. It then groups reviews answering a similar question (and all their studies combined) and displays them within a single interface. The system automatically detects when new reviews answering the question appear, making it possible to maintain an updated body of evidence (based on what has been included in reviews so far). It is expected that over 95% of reviews in Epistemonikos will have their primary studies indexed by the end of 2016, with several thousand reviews and studies organised by question by the same date.
Steps are also being taken to capture published, unpublished and ongoing studies not yet included in reviews from a wide variety of databases, registries and repositories, to identify questions not answered by reviews, and to create a map of questions compatible with different taxonomies and systems.
If everything goes as expected, in the next 5 years we should have a repository of all the existing evidence for each health question, continuously updated, and retrievable in seconds.
The final step to complete Archie Cochrane’s vision will be summarising all the existing information into a high-quality, up-to-date synthesis, acceptable to different end-users and integrated into information systems. Epistemonikos might play a role there too, but this will require a global effort by organisations like the Cochrane Collaboration, BMJ Clinical Evidence and probably others. It is almost impossible to say how hard this last step will be, but we now at least know the magnitude of the task that lies ahead.

 

Gabriel Rada is Associate Professor of Internal Medicine and Director of the Evidence-Based Healthcare Program at Pontificia Universidad Católica de Chile. He is co-director of the Southern American Branch of the Iberoamerican Cochrane Centre, editor for the Cochrane Effective Practice and Organisation of Care Review Group and member of the GRADE working group. He is also co-founder and president of Epistemonikos Foundation.

By submitting your comment you agree to adhere to these terms and conditions
  • Jo van Schalkwyk

    Gabriel,

    In my practice, I have learnt through bitter experience that if the assessment is wrong, it’s likely that there are also errors in the data recorded in the clinical notes. There’s an intimate relationship between the data accumulated and the cognitive model of the person doing the accumulation. This cuts both ways—not only do bad data lead to bad models, but bad models lead to bad data acquisition. This problem translates into the domain of research too.

    “Evidence” is not like gold, to be mined and accumulated in a vault, but depends on context and origin. The BMJ has been at the forefront of pointing out that some evidence is fabricated, some is withheld, some is of dubious provenance, and most is taken out of context.

    Perhaps I have this wrong, but what I didn’t get from the post is how you will integrate and weigh up all of the evidence.

    Without a robust system for evaluating quality, and integrating data from multiple domains, you may well mislead clinicians into sub-optimal decision making. How will you address this problem?

    As the traditional approach of “levels of evidence” has no basis in logic or mathematics, and is divisive rather than integrative, how will you optimise your decision making? (Bayesian or something else?)

    My 2c, Jo.

You can follow any responses to this entry through the RSS 2.0 feed.
BMJ Clinical Evidence Blog homepage

BMJ Clinical Evidence

Clinical Evidence is a database of systematic overviews on the effectiveness of key interventions, together with tools and resources to learn and practise EBM. Visit site



Creative Comms logo

BMJ Clinical Evidence latest news

BMJ Clinical Evidence latest news