By Maarten Oranje

In a recently published paper in BMJ Global Health, a group of authors (Paul et al) challenged overly optimistic views on performance-based financing (PBF) in the health sector, and its implementation in low- and middle-income countries. While some of the issues they raised are legitimate, the paper paints an overall picture of PBF that does little justice to the reality.

At Cordaid, a PBF-implementing NGO mentioned in the paper, we strongly believe in rethinking our way of working to make sure the needs of the most marginalized in this world are being met. We argue, however, that good rethinking can only start with a balanced presentation of the current situation. It is our belief that a more constructive debate is possible – a debate that goes beyond black and white and focuses on a slightly different question: in which form, and under which circumstances, has PBF been transformative? And when has it not been so?

Paul et al argue that PBF is not solidly rooted in evidence, it is expensive and that it produces only short-term results. They state that PBF is donor-driven, inflexible, and thus unsustainable. As a starting point for debate, we respond to these critiques and, in doing so, provide some suggestions for the way forward.

Evidence

The authors rightly claim that meta studies of incentive-based programmes often result in mixed or low certainty evidence. The same is true, however, for meta studies of many other health systems and health financing approaches. Why is that? Because all systemic interventions are ineffective? Or because we do not always use the right instruments to measure effectiveness?

We argue that this is partly because meta studies synthesise evidence and try to distil an overall effect from widely varying programmes – simple incentive schemes and system wide reforms – and in many contexts. Given that a typical PBF programme targets a broad set of quantity and quality indicators, it can be hard to substantiate the overall effects of one programme, let alone the effects of a set of programmes in many countries. In-depth case studies of individual programmes provide some insights which meta studies may not. We could also use more refined statistical methods, for instance adding Bayesian statistics to our current largely Fisherian toolbox. Rajtokia and colleagues have used other innovative methods to measure the effects of a PBF programme in Mozambique. The new tools developed by these researchers could be used to evaluate other PBF programmes, thereby building more systematic evidence of what works and what does not work.

Costs

As the authors correctly state, the verification and other operating costs of a PBF programme are substantial; up to 30% of the total budget. Verification efforts need to be intensive especially in the first years of implementation, to ensure data reliability and build data and management capacity of health staff. But after a few years, a lighter, risk-based verification mechanism can be introduced, as Cordaid successfully did in Zimbabwe, where we lowered verification costs by 47% in the first three years of implementation. Whether the costs of a mature PBF programme are high (typically $2–3 per capita per annum), is primarily a matter of perspective. We argue that it should be measured in terms of cost effectiveness, looking at additional costs in relation to additional health outcomes. Shepard, Zong and Nguyen have suggested refined ways to do so. In doing so, it is also important to consider that PBF expenditure does not have to be a top-up of ‘regular’ healthcare spending. On the contrary, integration of PBF principles into national health financing should be the ultimate objective, as has been done in Rwanda, Burundi, Armenia and Cambodia. And the cost effectiveness of future programmes could be further enhanced by more strategic purchasing of indicators.

Driving forces

The authors state that although PBF was originally conceived as an open and flexible approach, it is often implemented under a ‘mainstream, unsustainable, donor-funded and donor-driven’ approach. We agree that in some PBF programmes, there is room for improvement in contextualization and national ownership. The fact that an approach is not home-grown, however, does not question its legitimacy. Nor does it mean that it is only donor-driven. For Cordaid, PBF is a way to bring local stakeholders together in an inclusive process of health system strengthening. Whenever we start piloting PBF in a country, often without any support from institutional donors, we have observed local stakeholders becoming strong proponents. Take the example of the head of the Zonal Health Department in the pastoralist area of Borana, Ethiopia. During a Steering Committee Meeting in 2017, he stated: ‘In two years’ time, PBF has done more for the health of our people than the last twenty years of aid’. He had seen health staff regaining motivation and a sense of ownership, clinical conditions improving and patient flows growing rapidly. And now he is actively promoting PBF at higher levels of government. This example is of course anecdotal, but it demonstrates that PBF may be a truly transformative force, in specific contexts.

Sustainability

Which brings us to the final point of critique, that PBF only produces short term results, diverting attention and resources from the health system at large. Contrary to the claim of Paul et al., results of a full-fledged PBF programme are often not fleeting. To name just two examples, Rajtokia et al. found a large and sustained increase in Mozambique. And in Zimbabwe, performance levels did not drop after five years of PBF. This is true even  after taking away, or substantially lowering incentives, as has been seen in Argentina and Uganda. This seems specifically true where PBF was implemented in a full-fledged manner and not as a simple blueprint with a short-sighted focus on the payments. As a matter of fact, most recommendations made in the paper for ‘alternatives’, are addressed in well-designed PBF schemes. A transformative PBF programme improves working conditions, reinforces the self-esteem of health staff, utilizes financial rewards in remote areas, utilizes data for local decision making, increases community empowerment and reduces financial barriers. In studying PBF, we should clearly distinguish between simple pay-for-performance schemes and systemic interventions that follow the full PBF approach.

Conclusion

We have tried to identify some features of successful PBF programmes – full-fledged application of PBF, true local ownership, meaningful equity measures, strategic purchasing of indicators and an institutional set-up which anticipates later integration into the national health financing strategy. However, we agree that there is need for further research and therefore wholeheartedly embrace the exhortation, by Paul et al, for independent researchers: ‘(1) to continue studying what is happening in the field, while minimising the risk of producing biased results; and (2) to focus especially on how stakeholders transform the model in practice, how PBF changes their behaviour, and on PBF’s complex long-term effects on motivation, including the effects of its interruption following donors’ withdrawal.’ But in doing so, let us make use of all methods and instruments available, including operational data analysis, action research, impact evaluations etc. Let us jointly identify why some PBF programmes are highly successful, and others less so. Let us reduce fragmentation in the health system and find adaptable approaches that can successfully strengthen the system. Let us tap into the power and holistic approach of PBF and continue to examine under which circumstances it is effective, and in which it is not.

About the author: Maarten Oranje, MA, is an expert in Performance Based Financing and in the use of data for decision making in health systems. He is currently working at Cordaid, a Dutch NGO, and is a member of the UHC 2030 Technical Working Group on Support to Countries with Fragile or Challenging Operating Environments

Competing interests: I have read and understood the BMJ Group policy on declaration of interests and confirm I have no conflicts of interests to declare.