This blog is part of a series of blogs linked with BMJ Clinical Evidence, a database of systematic overviews of the best available evidence on the effectiveness of commonly used interventions.
Rapid reviews are increasingly commissioned, used, and written about. But why is there this—relatively sudden—interest? Putting it bluntly, it’s because the cornerstone of evidence synthesis, the systematic review, is becoming increasingly out of touch with the needs of today’s healthcare systems.
The journal Systematic Reviews recently featured an editorial “All in the Family: systematic reviews, rapid reviews, scoping reviews, realist reviews, and more.” In the article they report: “It is our view that all of these new forms of reviews are related to systematic reviews, similar to the way that different biological Species within the same Family are related to each other.” Tantalisingly, they later raise the issue of the extinction of some methods. Is it a given that systematic reviews, as we know them, deserve to survive?
For many years, systematic reviews have been relatively free from evolutionary pressure. The majority of changes have been internally driven, frequently relating to improving the methodological rigour of the product. But this focus on rigour has often been made in isolation of the ensuing effects on cost and timeliness. The reality is that systematic reviews are costly interventions that typically take over a year to produce and which are, frankly, boring.
Rapid reviews fill an evolutionary gap long vacated by traditional systematic reviews—meeting the dual requirements of speed and low cost. If a review is needed in a month and on a modest budget, a systematic review isn’t worth considering. The scenario of the need for a rapid and low cost evidence synthesis is hardly atypical and it’s a result of this that rapid reviews are on the rise.
However, rapid reviews are an ungainly collection of different techniques, ranging from Trip’s five minute system to those taking six months or longer. There is no coherent language to describe them, leaving users to ask for “rapid reviews” with little appreciation of the diverse methods that are available. Even more problematic is the lack of rigorous evaluation to underpin each separate method. Given the relative youthfulness of rapid reviews this is hardly surprising, and a number of groups are starting to put together research plans to better understand the promise and limits of the various rapid methods.
Assuming that this research helps qualify the usefulness of rapid review methods, there is the need to see them within the wider context of evidence synthesis. My own view is that there needs to be a more nuanced approach to reviews/syntheses—one that doesn’t start with the kneejerk “we need a systematic review” with no questions asked.
I’m increasingly drawn to the view that a rapid review is the default opening method for those interested in a synthesis of the evidence. A rapid review allows for a rapid overview of the existing evidence. And one would only proceed to a fuller review if certain criteria are met. For instance, are the outcomes useful, are the effect sizes likely to lead to change, and are we able to produce a fuller review within the timescale needed? Unless there is clear gain, it appears unethical to use valuable resource on a product that is likely to be of little use.
Rapid versus systematic reviews is a battle for position, a battle for resource, and a battle for influence. Some vested interests may rely on the status quo being maintained. Irrespective of this, this fight will ultimately be resolved by which method actually meets the user’s needs. As Voltaire said “perfect is the enemy of good,” and if the systematic review world unquestioningly follows the notion of perfect methods it’ll surely be the rapid methods that prevail.
Jon Brassey is the founder and director of the EBM search engine the Trip Database. In addition to this he works as lead for knowledge mobilisation at Public Health Wales; is an honorary fellow at the Centre for Evidence Based Medicine, Oxford; and recently started the Rapid-Reviews.info website.
Competing interests: I have read and understood BMJ policy on declaration of interests and declare the following interests: I run the search engine the Trip Database (www.tripdatabase.com) and own 50% of the shares in the company. One area Trip is actively researching is rapid reviews, and in the future this interest may be commercial in nature.