The World Health Organization has never been the most radical or dynamic body, which—as an international, public, administrative organisation with a ferociously complex governance structure—is not entirely surprising. But in its annual World Health Report of 2000, it did something that was in its own way quite revolutionary: it produced a ranking of the performance of the health systems of countries around the world (nearly 200). This turned out to be a brave, if not foolhardy, venture.

The countries at the top of the list—France and Italy came first and second—basked in the glory. Others, not quite so high up, tried to downplay or completely ignore the results and their significance. There is an anecdotal story that a UK health minister, on hearing about the UK’s position (18th) expressed surprise that the country had done as well as that. But other countries lower down the ranking list were not so happy. Many of them expressed their great irritation about the report, and the fact that WHO was denigrating their efforts and achievements.

Of all the countries that were upset, none was more so than Brazil. Brazilians regarded it as a national insult to be ranked only 125th best, just after Bhutan, and far behind South American neighbours, such as Colombia (22nd), Argentina (34th), Chile (33rd), and Venezuela (55th). Not only did the Brazilian government strongly complain about the report, but it set up a team of statisticians and other experts to go through its data and methods in detail, looking for inaccuracies and errors to undermine it. And, of course, they found a great many. (This was not, in fact, very difficult to do because for many countries there were very few data, and the rankings had to be based on a series of heroic assumptions using projections and extrapolations from what was available, including information from different periods and even from other comparable countries.) I was present at an extraordinary meeting that took place during the World Health Assembly in Geneva, when a team flown in specially from Brazil presented its—largely negative—findings to country delegates and rather embarrassed WHO staff.

After this political debacle, WHO steered well away from such comparisons for years. And many countries made it very plain that they did not welcome such international ranking exercises.

However, this position has begun to change. Under the twin pressures of economic constraints and growing demand, health services are eating up ever larger proportions of public spending. Consequently, there has been considerably increased interest in many countries in the functioning of their health systems as key parts of their national and regional economies. This means investigating and, as far as possible, measuring how efficiently and effectively health systems are functioning. To do this accurate information is needed, not just on what resources are involved—numbers of doctors, numbers of hospital beds, amount of drugs prescribed, number of operations performed, etc—which many countries already have large amounts of data on, but also on what is produced—the outcomes of health systems. Here there is considerably less information.

Some steps are being taken to address this. There are initiatives in many places to try to measure the results of particular healthcare processes and interventions, looking at both clinical outcomes and patient reported outcomes. In parallel, at the health systems level, Sweden has had for a long time a system of annual regional comparisons of quality and outcomes on the basis of a number of indicators, which draw on its well established disease registry system as a source of data. The Netherlands has also set up a national structure, again with a large number of indicators.

And in 2010 England set up an ambitious health outcomes framework, which differentiates between public health outcomes, health service (NHS) outcomes, and social care outcomes. Work to develop indicators that measure performance based upon data being collected in health and social care is still ongoing, but this could eventually produce powerful results.

In regard to international comparisons, since 2002 the Organisation for Economic Co-operation and Development (OECD) has been developing its Health Care Quality Indicators project, which contains some indicators on outcomes for its 34 member countries. The Commonwealth Fund, a US foundation, produces its own annual rankings of health system performance in some OECD countries using OECD figures and other data sources.

In the EU, the European Commission has gradually been establishing elements of a health information system for member states, but it has repeated a mantra for years that it was not going to start making comparisons between them. And its reluctance to get into this area was reinforced by the constant reminders from many member states that, although they welcomed having more health information, they decidedly did not want to see any production of league tables about how well they were doing compared to others.

However, more recently, the attempt to find ways to improve the EU’s growth and to strengthen the eurozone’s economic governance in response to the economic crisis has reopened the issue. One important tool that has been developed is the European semester. Under this annual process, member states receive recommendations about improving their budgets and economic plans. And, as health system expenditure is a large part of total public expenditure, most countries receive some recommendations aimed at improving their health system’s performance.

At the moment, the semester’s focus (and the recommendations it makes) is heavily one sided: it is mainly on tackling inputs—controlling or even reducing the resources used—rather than on improving the outcomes of the system. No doubt this reflects the prevailing view in finance ministries that health expenditure is a cost, not an investment. But, to some extent, it is also an inevitable reflection of the fact that there is much more information available on inputs than on health outcomes. Yet, regardless of what one thinks about the recommendations being made, at least the process has helped to put the measurement of health system performance back on the agenda.

Thus it is not a coincidence that the new European Commission president, Jean-Claude Juncker, highlighted the need for “developing expertise on performance assessments of health systems” in his mission letter to the incoming health commissioner. Already, in 2013, the Council (of Health Ministers) had invited the European Commission to support countries with using health system performance assessment (HSPA), and to work with member states to identify tools and methods, help select priority areas for HSPA at EU level, and improve data and information.

As part of this effort, in September 2014, an expert group on health system performance assessment was established, co-chaired by Sweden and the European Commission, and is now working on the issue. It remains to be seen what it produces in practice. The first report is due at the end of the year. But one thing is certain: whatever else this expert group does, it is not going to be in the “naming and shaming” business.

Bernard Merkel retired last year from the European Commission, where he played a central role in the development of the EU’s actions in health, including the public health programmes; the overall health strategy; and, more recently, the growing work on health systems. Before he joined the EU, he was a civil servant in the Department of Health for many years, including a spell as a ministerial private secretary.

Competing interests: I am receiving a pension from the European Commission and I will also be doing the odd piece of work for them, as well as some paid and unpaid health policy consulting, teaching, and research.