We may like to think with websites that allow us to compare prices and get feedback on books, plays, and restaurants that transparency is empowering us, but is the balance of information fair? Do we know more about Google, Tesco, and the government than they know about us? Clearly not, and, argued Roger Taylor and Tim Kelsey, in a lecture this week to launch their book Transparency and the Open Society, transparency has great promise but we have a long way to go to reach a truly open society.

The authors define transparency as “the degree to which I can evidence whether or not I am being treated fairly.” (This is the first time I’ve encountered evidence as a verb, but truly there is no noun that can’t be verbed.) Somebody in the audience suggested that the definition should mention quality as well as fairness: we might all be getting equally bad services.

There are four ways in which systems (governments, corporations, hospitals, and others) may diminish transparency. Firstly, they can simply fail to collect data. Secondly, they may redact information: the scandal of the expenses of members of parliament, which culminated in five being convicted, broke only when unredacted data were released. Thirdly, portals may present data in ways that are unhelpful or may even mislead. For example, portals on educational attainment showed that poor children and those from ethnic minorities performed poorly, but it’s only when data on ethnicity and poverty were combined that people could see that poor white children had the lowest performance. Fourthly, data may simply be withheld.

The authors identified four levels of transparency. In transparency 0.0 the citizen (or customer or patient) provides data to the “allocator” (government, corporation, hospital, etc) and gets nothing back.

With transparency 1.0 the citizen supplies data to the allocator, and the allocators give some account of themselves. The hospital might give you data on its waiting times, cleanliness, or even rate of wound infection, but the allocator (hospital) chooses what to supply—and comparisons with other allocators (hospitals) are probably not available.

“Challengers” enter at transparency 2.0. These are organisations that gather data from the allocators, analyse it, and present it back to the citizens and allocators to help them make decisions. Dr Foster, which was founded by Taylor and Kelsey in 2001, is the best known example of a challenger organisation within health in Britain. The government gave it access to hospital data, and it produced the Good Hospital Guide, showing big variations in avoidable deaths among hospitals. These data, they argued, led people to think not just about waiting times and things like parking at hospitals but also about safety. There have, of course, been intense arguments over the reliability and usefulness of Dr Foster data, but Taylor and Kelsey are not only unphased by the arguments but welcome them, seeing them as fundamental to democracy and understanding. (I couldn’t bring myself to use the word wisdom.)

Challenger organisations must be transparent themselves and must be able to keep data secure. Dr Foster has three components: an academic unit that analyses the data; a business, which was mainly a media operation—presenting and distributing the data; and an ethics committee, which is a separate entity, that makes sure that all releases of data are in the public interest.

In transparency 3.0 challengers have access not only to population data but also to individual data. For example, Ofgem, the British regulator of gas and electricity markets, compares the deal offered by different suppliers, hoping that customers who have bad deals will realise and switch to better deals. But if Ofgem had access to individual data as well it could communicate directly with individuals that they were getting a poor deal.

We might well need more transparency 3.0 challengers because we know, as several members of the audience pointed out, that people usually fail to respond to data. Famously, Bill Clinton when needing a heart operation went to one of the worst performing hospitals in New York. UnitedHealth (for which I once worked) had nurse advocates discuss with patients needing heart transplants data on the number of transplants done by individual hospitals and their outcomes. The result was that half of patients crossed state boundaries for their operations, which probably wouldn’t have happened if the data were simply shown on a website.

Kelsey spoke with great enthusiasm about the 311 system that works in New York and Baltimore. In this system citizens communicate in real time with the city authorities about things like potholes in the street, refuse collection, and much more. Some 80 000 citizens contribute data. These data, which are shared as well with third parties, have become the main way that the city authorities manage services and suppliers. Kelsey, who was a senior manager at NHS England, imagines something similar operating in the NHS—real time data on how long patents are waiting, the food they are receiving, whether doctors are keeping them informed, and much else.

By the end of the talk the audience seemed convinced that we still have a long way to go with achieving transparency that will be balanced between individuals and organisations and allow citizens to ensure that they are being treated fairly and with high quality services. How long it will take to reach that goal is unclear, but Taylor and Kelsey have mapped a path.

The lecture was provided under the aegis of the Imperial College’s Institute of Global Health Innovation. Transparency and the Open Society by Roger Taylor and Tim Kelsey is published by Policy Press, £23.99.

Richard Smith was the editor of The BMJ until 2004. 

Competing interest: RS is an adjunct professor in the Institute of Global Health Innovation and was once employed by UnitedHealth Group.