Rob Bethune is a surgical registrar in the Severn Deanery. Follow him on twitter – @robbethune
One of the mantras of the quality movement is copy shamelessly from others. This is valid and sensible; there is no point inventing the wheel over and over again (and more importantly not re-inventing the flat tyre). However there are two important caveats to this; make sure you know what you are copying and be prepared to adapt the projects to fit your local circumstances. This blog is brought to you by BMJ Quality. For more quality improvement resources go to quality.bmj.com
‘In the South Seas there is a Cargo Cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas – he’s the controller – and they wait for airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things Cargo Cult Science, because the follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.’ – Richard Feyman[1].
The clear translation for us is that if you superficially copy a quality improvement programme you may well fail. This argument is expanded in this excellent article by Mary Dixon-Woods that evaluates in detail why the Michigan central line infection programme worked. As it turned out it was not just about the five point checklist (and in fact by the end there were over 100 different locally adapted checklists) but a complex social intervention [2]. The cocktail hour in the evenings of the collaborative meeting was seen as crucial in creating the culture to drive improvement. Details are really important; they also had logos for all the participating ICUs printed onto the water bottles at the joint events; that is how you change culture.
As a surgical registrar I see this cargo cult style error daily. In almost all theatres across the UK the WHO safer surgical checklist is used before, during and after each operation; well at least that is what the data from trusts will show you. A study was published in the NEJM that showed the benefit of this checklist and the idea was that if all UK hospitals used this checklist then they would get the same reduction in mortality and morbidity[3]. However in the same way the islanders superficially copied the Americans with their wooden huts and bamboo ear defenders we have superficially copied this quality improvement intervention. The NEJM study did not just use a checklist they had pre-operative briefings and crucially team training on how to use the checklist. There is a whole science around checklists and a study in the British Journal of Surgery shows that we are simply not using it properly and therefore cannot expect to get the same benefits as the NEJM study showed[4]. We may laugh at the islanders but we are doing the same thing on mass in the NHS.
Just because a quality improvement intervention worked in one place does not automatically mean that it will work in your clinical area. In fact it is likely that without some alterations to fit your local circumstances it won’t work as well. In this paper (co-authored by @carsonstevens) this point is expanded on – if you are not prepared to adapt work from elsewhere and be flexible you are much more likely to fail and this is one of the reasons why so many QI initiatives do not succeed[5]. Again we have not heeded this advice during the roll out of the WHO checklist. It must be adapted for local situations. Some of the questions are simply not relevant to UK hospitals and equally there are other questions that need to be included. Pilots do not use the same checklist for each plane they fly and so we should not be using the same checklist for day case eye surgery as for an anterior resection for rectal cancer. If we become more flexible with the implementation we will get more staff on board and begin to effect the cultural change that was behind the original success of the safer surgical checklist.
So if you want to avoid being one of the islanders with bamboo headsets, before you start to implement the next amazing improvement initiative make absolutely sure that you know in detail what you are copying and be prepared to be flexible in the implementation.
1. Feynman RP, Robbins J. Cargo Cult Science: Some Remarks on Science, Pseudosciene, and Learning How to Not Fool Yourself. The Pleasure of Finding Things Out. Cambridge, MA: Perseus Books, 1999:205-16.
2. Dixon-Woods M, Bosk CL, Aveling EL, Goeschel CA, Pronovost PJ. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q 2011;89(2):167-205 doi: 10.1111/j.1468-0009.2011.00625.x[published Online First: Epub Date]|.
3. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360(5):491-9 doi: NEJMsa0810119 [pii]
10.1056/NEJMsa0810119[published Online First: Epub Date]|.
4. Pickering SP, Robertson ER, Griffin D, et al. Compliance and use of the World Health Organization checklist in UK operating theatres. Br J Surg 2013;100(12):1664-70 doi: 10.1002/bjs.9305[published Online First: Epub Date]|.
5. Parry GJ, Carson-Stevens A, Luff DF, McPherson ME, Goldmann DA. Recommendations for evaluation of health care improvement initiatives. Acad Pediatr 2013;13(6 Suppl):S23-30 doi: S1876-2859(13)00099-5 [pii]
10.1016/j.acap.2013.04.007[published Online First: Epub Date]|.