Parliament has three times relegislated the commissioner provider split—in 1990, 2002, and 2012, said Stephen Dorrell, secretary of state for health from 1995-97, in a talk to the Imperial College Centre for Health Policy this week. Every health secretary for the past 26 years—with the exception of Frank Dobson—has believed in commissioning. But, he asked, how does it work, how should it work, and how might it work in the future?

The concept, Dorrell said, is simple: the commissioners should start not with existing institutions and ways of doing things but with the needs of individual citizens. They should then set priorities and find the best way to meet those needs paying attention to quality and value for money. They should be open minded and not bound to existing providers. Commissioning, argued Dorrell, provides our best chance of reshaping the care system, which we badly need to do and have so far failed to do.

Commissioning empowers local authorities who better know the needs of local people than the centre and who if genuinely empowered will do a better job than central planners. The alternative to commissioning is central planning, and, said Dorrell, one of the big lessons of the 20th century is that central planning does not work. Yet when the NHS is funded from income tax there must be a means to hold local authorities to account, and commissioning provides that means.

So commissioning works in theory and is such an attractive concept that all secretaries of state for health (bar one) for the past 26 years have opted for it. But does it work in practice?

Dorrell articulated that question and its sister “Does competition work?” He acknowledged that many people have analysed the evidence to give different answers but didn’t attempt to summarise the evidence. But he did say that competition is not a choice but a fact of life. He was, he said, competing as he spoke, putting forward an argument when others would have a competing argument. There is always competition of ideas and for resources and attention.

In effect, however, he accepted that commissioning in the NHS has failed—it has been too process driven, too dominated by lawyers, and hasn’t delivered the system change that’s needed. But he was convinced that we should not go back to central planning, which means the system doing the best it can and explaining why that’s all that can be done. The “fatal flaw” in the system, he claimed, was that the commissioning was for the NHS (the “National Sickness System”) rather than for the totality of care. Those who commission care for elderly people should, for example, be able to commission social care and social housing, so avoiding unnecessary admissions to hospital. Those commissioning children’s services should be linked to schools and the criminal justice system—and so able to break the cycle that leads to truanting, drug addiction, prison, and alcoholism. The children who fall into this terrible trap cost public services a fortune but receive little benefit.

The “fatal flaw” has been recognised, and one result is to give the city of Manchester its NHS resources and allow local authorities to commission from all public services (which Dorrell defines as service for the public rather than services owned by the public).

Dorrell outlined five conditions that should allow commissioning to work properly: commissioners should:

• Put health before illness. At the moment, almost all NHS expenditure is on illness services.
• Focus on individuals in their social context and “support normal life.”
• Create rational incentives; those that we have at the moment are far from rational.
• Accept responsibility for joining up services.
• Commission for a place and a people.

Success with commissioning will also depend on national politicians being able to resist the temptation to meddle, and Dorrell accepted that this hadn’t happened so far. Whenever there’s a problem, he said, politicians send for “an inspector with a clipboard,” a response that doesn’t work. Why, he asked, did the commissioners in Mid Staffs get off Scot free?

Those in the audience questioned whether the Manchester experiment will work. Where will the competition come from? Or will authorities in Manchester degenerate into a complacent monopsony? Are silos inevitable to deal with the complexity of the modern world and is joining up services just too hard? Do commissioners have the necessary competence? Dorrell accepted that commissioning is difficult and that given a choice between running a hospital or a commissioning authority able managers have usually opted for the hospital. Will joined up commissioning inevitably mean less funding and fewer staff in the NHS and will that be politically acceptable? Dorrell thought that it wouldn’t lead to a diminished NHS.

The Manchester experiment is exciting, but is it a high risk gamble that may lead to triumph or disaster? Or will the NHS “muddle through” in its traditional way? We must wait and see. It may be a long wait.

Competing interest: The Centre for Health Policy is within the Institute of Global Health Innovation, where RS is an adjunct professor (as is Stephen Dorrell).

Richard Smith was the editor of The BMJ until 2004.