An ugly outcome
In the early hours of Friday 28 June, a surgeon lay awake with worry. Finally he decided to go into hospital to catch up with his paperwork and emails. As he walked down the hospital corridor, a nursing colleague saw him and gasped, “Have you seen the news?” before giving him her smartphone.
Above a grainy photograph of his face, the headline read, “The surgeons whose patients were up to 30 times likelier to die.”
“I felt like I had been kicked in the stomach,” Simon Payne says.
Simon Payne is a consultant vascular surgeon. On Friday, raw mortality rates for individual consultants in his specialty were being published for the first time. Two operations were used to classify their performance: abdominal aortic aneurysm repairs (repairing abnormal swellings of our largest artery, the aorta, in the abdomen) and carotid endarterectomies (unblocking one of the arteries to the brain). Although only raw mortality rates were being published, it is widely acknowledged that risk adjustment is needed for comparisons between surgeons.
Payne had performed 98 carotid endarterectomies with an unadjusted raw complication rate of 2.0% for stroke or death—he had performed the second highest number of operations in his region over the three years in the audit and his risk-adjusted complication rate was below the national average. However it was different for aortic aneurysm surgery:
“I had stopped doing aortic surgery in 2011. Some years ago, I had fractured my left arm badly and had had shoulder surgery for it. I found it was sometimes painful during aortic surgery and decided to concentrate on carotid surgery. But the 16 patients who underwent aneurysm repair had a higher risk and if you correct for the risk factors, there was no concern. When the Vascular Society of Great Britain and Ireland asked for my consent to data release, I gave it for carotid surgery, but pointed out that the aortic surgery data was not relevant to my current practice.”
Despite this, his aortic surgery data was included in the first release of the audit table. Of his 16 patients before 2011, five had died, giving the 31% raw complication rate. After risk adjustment by the Vascular Society for factors such as age, co-morbidities, and aneurysm severity, this was not significantly higher than for other surgeons.
On Thursday evening, Payne received two missed calls from his hospital medical director and chief executive. His medical director warned him that the aortic surgery rates were being published the next day. As he lay awake that Friday morning, the BBC Today programme had called Payne’s hospital asking to speak to the chief executive for an interview about Payne’s performance.
Journalists continued to call his hospital repeatedly throughout the day. The Daily Mail published a photo of him; the Daily Telegraph ran the story on the front page; and the Independent listed his medical school, his pastimes, and patient comments.
After Payne spent most of Friday in meetings with his hospital’s management, the Trust released a statement of support and the Vascular Society issued a revised table excluding Payne’s name from the list of consultants performing abdominal aortic aneurysm repairs.
Good outcomes
On air that Friday morning, the BBC Today programme made no mention of Simon Payne or his hospital. Instead the presenter asked Bruce Keogh, the national medical director of NHS England, why outcomes were being published.
“What I really want is for the patients to have the best possible outcomes. When a patient agrees to have an operation, it is the patient taking the risk,” he said. “I think we’re getting to a position now where the surgeon shares some of that risk with the patient.”
This is not new. In ancient Babylon, the code of Hammurabi stated that if a surgeon operated on a free man and killed him, then the surgeon’s hand should be cut off. Florence Nightingale measured surgical mortality rates during the Crimean War.
Bruce Keogh is a professor of cardiac surgery, a specialty that has been leading the way on publication of outcomes. In 1989, the New York State Department of Health began collecting data on the outcomes from coronary artery bypass grafting throughout the state (covering three quarters of all such operations). Risk-adjusted mortality rates were provided to hospitals for quality improvement programmes. From 1990, the data were released to the public with hospital level results.
Soon after, a newspaper sued them for access to surgeon level data. In December 1991, after winning the lawsuit, the Newsday paper published surgeon specific rates to great anger from surgeons. In following years, a compromise was reached: only individual data for surgeons performing over 200 CABG operations per year would be published. There was no evidence that patients had moved away from hospitals or surgeons with higher mortality rates, and no evidence that surgeons had avoided operating on high-risk patients.
Instead there was a 41% decline in risk-adjusted operative mortality over the first three years—compared to a possible figure of 18% in other states (although of course, it is harder to tell since data were not routinely collected elsewhere). Since then, there has been inconsistent uptake of public reporting in the USA and multiple approaches to surgical improvement have been debated.
Fifteen years after the New York experiment, a similar scenario occurred in the UK. After a public inquiry into mortality rates from children’s heart surgery at Bristol Royal Infirmary, publication of hospital-level data was recommended across the UK. In 2005, the Guardian newspaper asked for individual surgeons’ outcomes using the Freedom of Information Act. From then on, about 80% of cardiothoracic surgeons have had their data published by the Society for Cardiothoracic Surgery (SCTS)6, spearheaded by then president, Bruce Keogh.
Ben Bridgewater is the current SCTS database chairman, professor of cardiac surgery and director of the Healthcare Quality Improvement Partnership (HQIP). He thinks that in the UK there have been similar results to the earlier US effort.
“It’s clear that the mortality rates in cardiac surgery are now a third of what they were ten years ago, so that’s a very big effect. What leads to the improvement is professionals engaging with national clinical audit data so that they can benchmark themselves and have a relentless focus on the quality of the outcomes,” he states.
HQIP has been supporting and managing the roll out of individual surgeons’ outcomes in nine other specialties this summer. In each specialty, only certain procedures have been included initially. For example, in vascular surgery, patients who undergo carotid endarterectomy typically do not need extensive post-operative management and stay in hospital for only one or two days afterwards. Therefore a surgeon’s technical skill is thought to be one of the main determinants of post-operative stroke or death.
Recently there has been much attention devoted to surgeons being able to opt out of individual reporting. Bridgewater explains, “The data has a consultant name attached to it, and that is personal information as defined by the Data Protection Act as I understand it. When we first took advice we were told that actually just publishing without having consent of that individual was at odds with data protection. We got that from two separate bits of legal advice, one from HQIP and one from NHS England. So we opted to take consent. Something like 96% of the people who responded gave consent.”
If those who withheld consent were “outliers” with higher than expected complication rates, Bridgewater is confident that the personal right to data protection is outweighed by a public interest clause in the Data Protection Act. In the event, no surgeon who withheld consent was an outlier and nobody was forced to release their data.
See also:
Krishna Chinthapalli is the BMJ clinical fellow.