Hilary Thomson tells us more about the SWiM project team’s development of the SWiM guideline for the synthesis of quantitative data of intervention effects
My co-authors and I are delighted at the successful completion of the SWiM (Synthesis Without Meta-Analysis) project. From my own perspective, there is also a huge sense of relief. The SWiM work has involved working with some fantastic methodological experts from around the globe, but it also involved grasping a very large nettle which many before us had avoided, and I can now see why!
SWiM originated as a project to address serious shortcomings in the reporting and conduct of narrative synthesis of quantitative effect data (ICONS-Quant) in systematic reviews. Despite frequent use and longstanding concerns about the validity of narrative synthesis, there has been almost no methodological development to promote clearer methods for narrative synthesis since guidance was first published over 10 years ago. This methodological neglect has served to perpetuate the problems with these reviews and suggests that narrative synthesis is a dirty little secret, even taboo, among the evidence synthesis community. The issues covered by the SWiM reporting items may appear blindingly obvious, and the alternative methods of synthesis that we refer to are not highly technical. However, our previous work establishing the mess that is reported as narrative synthesis in nearly half of health related systematic reviews, suggests that this has not been obvious enough. SWiM embraces the key underpinning principles of a scientific approach to evidence synthesis and, as we point out, may have relevance when synthesising intervention effect data regardless of the synthesis methods used. It is, therefore, ironic that the fundamental principles of scientific rigour and transparency in synthesis have been so neglected in favour of developing a range of highly technical additions to the suite of meta-analysis when SWiM is so commonly used. It would seem that running before walking is being encouraged in some areas of the synthesis community.
Of course, grasping a nettle involves getting stung. One of the trickiest issues for SWiM has been around terminology and a definition of narrative synthesis. We had many lengthy discussions about this, but with little conclusion beyond agreeing that the term “narrative synthesis” perpetuates ambiguity and confusion for authors and readers. Putting that difficulty aside, we moved on to developing the reporting guidelines based on an expert consensus exercise in the hope that this may lead to improved clarity about what authors understand by the term “narrative synthesis,” and what it involves. One year later, while we were becoming clearer about the methods that are used in narrative synthesis of quantitative effect data, we were no further forward with terminology. The SWiM acronym is a bit problematic. It is an umbrella term for a range of synthesis methods, and is a negative definition, defining it by what it is not (Synthesis Without Meta-Analysis). I also recognise the potential oxymoron in this acronym: implying that synthesis is not meta-analysis (this still stings a bit). However, recognising the urgent need to address the near zero transparency in “narrative synthesis,” and that we had exhausted our discussions, we settled on SWiM.
Despite these stings, I am very relieved that we have been able to produce consensus guidance that has been well received by international expert peer reviewers on a methods topic around which there is much disquiet but little open discussion. SWiM will not be the last word for narrative synthesis. SWiM has only been able to address a small part of what may be contained in the term “narrative synthesis.” With calls for systematic reviews to incorporate more diverse evidence sources and address questions that respond to questions about complex policy interventions, we expect that SWiM will be relevant to a growing number of reviews. For now my hope is that, in addition to improved transparency in reporting, the SWiM paper will trigger more open discussion about the use and implementation of SWiM approaches.
To maintain current interest in this topic, we have set up a SWiM virtual network to facilitate discussion and shared learning about SWiM. In addition, Cochrane Training will be hosting two webinars on SWiM, and an online training module. Please visit the SWiM website for more information on how to join in these activities and for more nettle grasping!
Hilary Thomson is the principal investigator of the Synthesis Without Meta-analysis project. No competing interests. Follow Hilary on Twitter @HilThom123