Tracey Koehlmoos: Bringing systematic reviews with a development focus to South Asia

Tracey KoehlmoosA substantial body of evidence exists to answer many of the questions asked by policymakers and development partners in low and middle income countries (LMIC). However, evidence is often scattered, inaccessible, and rarely presented in a form that provides an indication of the quality of evidence. Systematic reviews in all sectors have the potential to contribute to improving the evidence base from which policies and interventions can be developed and implemented. The principles that underpin the international effort to prepare and use systematic reviews of controlled trials for evaluating clinical practice, particularly in high-income countries can be similarly applied to the challenges of poverty reduction and development in LMIC.

With this in mind, in early 2012, we were fortunate to engage in new work with the objective of providing capacity building in systematic review methods using training, workshops, on-line training and support, and quality assurance that target researchers in the South Asia region.

From the proposal development stage until today I have felt that the Centre for Systematic Review at ICDDR,B has been fortunate to build a collaboration with the Campbell Collaboration, the Social and Public Health Sciences Unit of the Medical Research Council UK (SPHSU MRC), the Campbell and Cochrane Equity Methods Group and the International Initiative for Impact Evaluation (3ie). I feel as if my small Centre for Systematic Review stands on the shoulders of giants like pioneering methodologists Terri Pigott from Loyal University of Chicago and Hilary Thomson of the MRC, as well as experienced reviewers from the Equity Methods Group and awesome search librarian Jessie McGowan of University of Ottawa. 3ie’s global network, and support of developing country review teams has added unanticipated layers of context, reach, and real world urgency to the project.

Further, the funding for this work has come from the Department for International Development (UK). Our programme manager, DfID’s man in India, is the excellent Guy Howard, who leads the South Asia Research Hub. He has functioned more like an additional partner to the project (than, say, an absentee funder or an evil task master).  Guy is a researcher at heart and shares my beliefs that in addition to the knowledge translation benefits of having developing country reviewers engaged in developing country review topics, there are a host of benefits that come from learning and applying the methodology. For example—the creation of a stronger group of primary researchers in the region who understand the primary study design, a more global awareness around pertinent issues, and even more so an understanding of where there is evidence for development investment and where there is the need for further investigation.

Over the past year, this funding was used to develop, deliver, and implement:  i) a programme of systematic review training in South Asia; ii)  guidance for development partners on formulating review questions; and iii) systems of remote partnership, networking and support to regional review teams. If you are interested in viewing some of the course material it is available in the Centre for Systematic Review webpages.

Some of the material is lent to us directly from the Campbell Collaboration and from the Equity Methods Group, but there is also original material on complex interventions developed by Hilary Thomson of the MRC and a few sets of lecture slides that were developed by my team in Dhaka.

The course has been offered in Bangladesh (June 2011), India (January 2012), and Nepal (April 2012) (everyone will remember that I stayed close to home for a few months after the Colonel died). We have trained 81 students from across the globe. The focus has been on South Asian researchers but we’ve included students from Namibia and Thailand who are engaged in reviews but would have benefited from a little more methodological training and also a few DfID representatives from Glasgow, London, and Kinshasa who wanted to learn the methods but also came to the course thinking about how systematic review could be applied to evaluating the broader portfolio of programmes. It is worth noting that because our partners at 3ie advertise the course to their global network of partners, we have had a demand for the course from across South East Asia, the UK, and especially from the Horn of Africa.

Feedback has been overwhelmingly positive and the course has been adapted over time to focus on mixed-methods, logic models, equity and evidence to policy, and narrative synthesis. In order to avoid the zombie-induced boredom of four days of listening to review methods, we have worked as a team to develop and then incorporate group practical exercises into each half-day of the training. It has been a challenge to incorporate reviews from across the disciplines so that we do not just focus on our natural haunts of health or education, but really reach out for regionally relevant topics like environmental evidence.

One year into the project, I can say that South Asian researchers are being equipped to undertake systematic reviews to help inform decision making in different sectors of development. However, the success of the project will not be measured in how many bodies sit in the classroom, but rather by the number of regional review teams who receive funding or otherwise complete reviews. In order to facilitate fledgling researchers in the region, we will attempt to create a match-making service to link regional researchers with more experienced developed country reviewers. We will be able to start capturing these sorts of results after two forthcoming rounds of systematic review request for proposals.

Tracey Koehlmoos is programme head for health and family planning systems at ICDDR,B and adjunct professor at the James P Grant School of Public Health, BRAC University, Dhaka, Bangladesh.