The First Global Symposium on Health Systems Research ended on a crescendo of upbeat promise on the 19th of November, but amid all of the excellence of organisation, content, energy and enthusiasm, there was a murmur of methodological tension between the positivists and social scientists.

I know that while I was prepared for strong debate around the methodological issues of health systems synthesis, which emerged after my first presentation entitled “ Beyond effectiveness: What other kinds of knowledge do policy makers need and how should it be synthesised?”  A Scandinavian man whom I did not know, made the statement that almost all questions were questions of effectiveness. He further stated that to discuss synthesis on questions other than effectiveness showed a weakness in the designs of the primary research.

I feel now that during the session I may have dismissed this comment too lightly. The participants had asked multiple questions, and I think the panel wanted to capitalise on the general agreement on the major issue in the session, which was about the need for an international collaboration for health systems synthesis. To be honest, the statement of everything being effective seemed like one of those over the top ridiculous statements that did not call out for an immediate response.  I live in a research and decision-making reality in which questions other than effectiveness are most often asked.  Perhaps it is difficult for me to imagine a reality in which issues of equity, economic impact, quality, enumeration, context, and process are just an aside to “Does it work?”  Clearly, I missed the opportunity for some meaningful methodological discussion.

However, on Wednesday a woman with whom I have collaborated on a large set of multi-country case studies that was to be presented the next day, ended up sitting alone while dining next to a table of men whom she did not know. The small group of men was loudly dogging the lack of rigour in health systems research. She was so alarmed because this was an affront to the development of methodology for case studies on which she is still working and she was concerned that they would appear at the session and condemn the methodology. She could not eat and raced to her room and called one of her senior colleagues. The senior colleague went down and confronted the men–or actually, sat down with them to buy a drink and discuss their issues/problems with the methodology.  Turns out, the group included the same person from the health systems synthesis session the previous day. Also, perhaps because there were twelve parallel sessions, this group did not attend the dissemination of the country case studies.

No doubt the tension between those who believe that everything can be boiled down to a p-value versus those who study the “how” and “why” will serve the greater good of the field. Health systems research is multi-disciplinary and still a relatively young field. Debate and discussion should encourage scientists within the field to develop a hierarchy or taxonomy in which different types of questions in primary research can be answered with a prescribed set of designs—and those study designs should be developed alongside tools to measure their quality.

It seems unlikely that these issues will be allowed to fester until the next symposium. However, how advanced will the taxonomy and methodology become between now and then? An international  health systems researcher association (IHSRA?) likely will emerge from this meeting with a focus both on the systems research but also on the goal of achieving universal coverage.  This issue of methodological rigour will no doubt be a priority for the new group.  As for the next meeting, the Chinese have graciously offered to host, so maybe somewhere in China in 2012 or 2013.

Tracey Koehlmoos is programme head for health and family planning systems at ICDDR,B and adjunct professor at the James P. Grant School of Public Health, BRAC University, Dhaka, Bangladesh.