In my last blog I described my time in a goldfish bowl with some 35 GPs on a leadership course—how the process worked, and what I learnt about myself, and how GPs think about me. But what I really meant to describe was our discussion on commissioning, how to do it well.

In particular, I found myself warning against “getting too close to providers.” As I explained in the last blog, one of my many faults is that I don’t know what I’m going to say until I’ve said it. One of the GPs in the “outer ring” of the goldfish bowl picked me up on this: “Did you really mean it?” It sounded so aggressive, contrasting with all the cosy talk of integration.

On reflection, I think I did mean it. Many people don’t like the idea of a commissioner-provider split, but the problem with a “provider led” health service is that the providers will do what seems right to them regardless of what might be best for patients or the population. Indeed, providers don’t tend to think about the population, it’s an abstraction. Rightly they think about the patients in front of them—not necessarily understanding exactly what is best for them but certainly not understanding the needs of many patients that they never encounter.

So the job of commissioners—simple in concept, impossible to do perfectly—is to get the maximum health benefit for the population with the resources available. I think, and most of the GPs seemed to agree, that if you were trying to achieve that aim starting with a blank sheet you wouldn’t design a system like we have now—top heavy with hospitals and specialists. You’d aim for a system that put public health and prevention ahead of treatment, where most care was provided in the community, teams were more important than doctors, care was integrated rather than fragmented, and patients at the end of their lives died with dignity in their own homes not in intensive care units.

Many providers, suc as hospital specialists, may understandably not share this vision—for both noble and less noble reasons. As an oncologist concentrating on the needs of your patients you may legitimately want to spend large sums of money on expensive drugs to extend patients’ lives by weeks.

Who is to say no to them? GPs are clearly in the strongest position. Unlike public health doctors who deal with the abstraction of the population, GPs have patients, they are clinicians. They have a clinical and moral authority to say, “I too care about the patient who wants the expensive drug. She’s my patient. But I have more than a thousand other patients with other needs, not perhaps quite as pressing as our patient with late stage cancer but just as important.” It’s almost impossible for managers to carry the day in such circumstances. (And the expensive drug is, of course, the easy case. Most commissioning decisions are muddier and more complex.)

So I suggested that GPs as commissioners did need to be close to providers, listen to them regularly, understand their views as well as possible, work with them to improve services, but ultimately keep some distance, not be “captured” by them.

GP commissioners do need to be very close to their communities, much closer than to providers. They have a head start over managers in that they are in daily contact with local people and have greater respect. But they cannot rely on authority. If they think it best to shrink a hospital and shift emergency services elsewhere they have to convince the population that this is the right thing to do. This can be exhausting and difficult work, and they won’t convince everybody. Even when the broader community understands the case then there will still be some with banners in the street, and perhaps some of those banners will say something highly unpleasant about the GPs.

I emphasised to the GPs that I didn’t have answers to their difficult job of commissioning, but I said how I felt commissioning—in the sense of radically reshaping services—had hardly happened. So far it’s been mostly a bit more of A and a bit less of B.

One reason for that is a lack of good, real time data. If you want to reshape a service you need good data and evidence on what might be, and even more importantly data on what is happening now, how effective it is, and how much it costs. Without such data you cannot know whether things are getting better or not. Data are not a sufficient condition for good commissioning, but they are an essential condition.

I stepped out of the goldfish bowl glad that it wasn’t my job to reshape services but urging the GPs to relish the challenge. Would it make any difference if the session had never been? Probably not, but you never know. Sometimes, just sometimes, an acorn is planted and years later a fully grown oak offers shade and more acorns.

Competing interest: RS works for UnitedHealth Group running a philanthropic programme in low and middle income countries. UnitedHealth UK offers commissioning support services to the NHS, but RS has no responsibility for the UK company.