On 15th September 2016, a letter appeared in The Guardian with an unlikely collection of signatories including Lords, an MP, a number of medics, a leader in evidence based medicine and the physicist Professor Stephen Hawking. [1] The letter drew attention to the “freshly disputed evidence surrounding a weekend effect”, suggesting that claims for this apparent difference in mortality rates between patients admitted at the weekend and those admitted during the week are not supported by reliable research, and that “Critically… at least 13 independent, peer-reviewed papers were available to the secretary of state that refute his definition of a weekend effect.”

It is not surprising that of the hundreds of peer reviewed papers written on the “weekend effect”, there are at least 13 which might show something to the contrary. Professor Hawking will be very aware of the February announcement of the successful detection of gravity waves caused by the collision of two black holes, reported by the Laser Interferometer Gravitational-Wave Observatory (LIGO) this year. [2] To find this phenomenon, first predicted by Albert Einstein a 100 years ago, you need a sensitive enough detector to pick up these minute variations in the very fabric of space-time itself. Earlier attempts to detect them were stymied by a lack of sensitivity in the experimental design. [3, 4] Fortunately, in order to be able to observe a “weekend effect”, you don’t need anything like the resources at the disposal of the team at LIGO. What you do need is a large enough sample of patients to have a study sensitive enough to be able to detect a difference in the probability of death between those patients admitted at the weekend compared with those admitted during the week. Simple power calculations suggest you need at the very least around 70,000 patients or more to be able to detect a 10% higher risk of death in emergency patients admitted at the weekend.[i]

Although the 13 papers are not directly referenced in the letter to The Guardian, one of the signatories is a member of ”Justice for Health”, a group of doctors “speaking on behalf of the public” whose case for the recent judicial review,[5] conveniently lists them. Of the 13 cited, two of them don’t provide any estimates of risk of death [6, 7] and a further two include night time admissions in their definition of off-hours.[8, 9] If you include these last two, of the 11 studies giving an estimate of risk of death at the weekends (or off-hours), only two are above the magic figure of 70,000. [10, 11] Seven of the quoted studies do actually find a higher mortality at the weekend,[8-10, 12-15] but in most cases the confidence intervals are simply too wide (Figure 1). 8 out of 11 of the studies are consistent with an effect of 10% [8, 9, 12-17] and the rest are consistent with an effect of 5%.[10, 11, 18] The largest study finds a “weekend effect” but concludes that it may be a “statistical artifact”, caused by higher admission thresholds in A&E with sicker patients being admitted to the wards.[10] Although this is certainly a plausible explanation, on detailed reading, the paper presents no evidence to support these conclusions. One of the tables in the paper even suggests the opposite, that some groups of patients admitted at weekends are actually less sick. An alternative explanation for the smaller proportion of patients admitted from A&E at the weekend is that there is a higher proportion of patients at the weekend, unable to see their GP, attending with minor ailments, who are less sick and simply don’t require an admission. Another paper suggests the weekend effect can be explained by poor data quality. [16] Using data from nine general practices for stroke patients admitted to a single hospital trust, it examines electronic administrative records selected for a range of diagnosis codes (ICD I60-I68), many of which are not stroke, as well as non-acute (planned) admissions. After validating these with clinical records, the study finds, not unsurprisingly, a range of diagnoses, many of which are not stroke and a considerable proportion of non-acute (planned) admissions, with a bias towards weekdays. The real surprise is that the authors claim that this is evidence of inaccurate coding. It may also be worth noting that the “weekend effect” is likely to differ by institution and/or condition, and this study, like other studies [8,9,13,14,15,18] based on just one or two hospital trusts may not be generalisable to the country as a whole.

Faced with an ever increasing number of papers (some of high quality, some not) from which both sides of the argument are able to cherry pick, there is an obvious research tool that can help. The systematic review and meta-analysis provide a transparent and reproducible means of finding relevant papers and a method to pool the results of better quality studies to give an overall estimate of effect, getting around the problem of small studies and eliminating poor quality evidence. They are by no means perfect instruments, but are supposed to be the cornerstone of modern evidence based medicine. Are there any systematic reviews and/or meta-analyses of the “weekend effect” out there? Well, it so happens there are. At the time of writing, there are at least three published systematic reviews on the weekend effect, with several more in preparation. The first looks at outcomes in patients with acute myocardial infarction, encompassing 48 studies enrolling a total of 1,896,859 patients.[19] The combined estimate for the 16 studies looking specifically at the weekend suggests a significant effect for weekends compared with weekdays with an odds ratio of 1.06 (95% 1.03-1.09) for 30‑day mortality. A second systematic review looks at off-hour presentation in patients with acute stroke.[20] It includes 21 studies with a total of 1,421,914 patients and gives an overall effect for off-hours admissions relative to weekdays with an odds ratio of 1.11 (95% 1.06-1.17). The third published meta-analysis includes 140 studies (252 cohorts) and covers 28 specific diseases with a total of 25,450,474 participants.[21] The extent of off-hour effects varied across different diseases, but overall, off-hours is associated with increased mortality (odds ratio 1.11; 95% 1.10-1.13). As well as providing analyses by condition, for a subset of these conditions, the study compares international differences, administrative data vs. clinical data and focuses on weekend comparisons. The authors note that associations varied substantially for different diseases.

There is clearly a need for more research to be done. There is indeed a paucity of evidence on the relationship between staffing levels and the weekend effect, and to date, no published systematic review on this relationship. There is only one paper as far as I can find on junior doctor staffing levels, and interestingly, this US paper concluded that patients admitted to hospitals with more resident trainees had significantly higher mortality following a weekend admission, although the authors are at pains to stress they do not show a link between the care provided by trainees and patient deaths.[22] There is also an opportunity for more research into interventions to reduce the weekend effect and the role population and community factors, particularly primary care, have to play. Much focus has been on mortality, for the understandable reason that it is extremely important, and because a death is relatively easy to diagnose and is well recorded. However, mortality is a crude outcome and further work (and systematic reviews) could focus on variations in morbidity, pain, suffering and discomfort. Most importantly of all, we should disassociate the question of whether there is a weekend effect and what might be the causes from industrial disputes and political arguments. The medical profession, perhaps more so than politicians, should practise what we preach around the use of evidence.

Finally, as an aside, what would we find if we were to carry out a meta-analysis of the papers allegedly refuting a weekend effect? A random effects meta-analysis of the 11 papers that provide an estimate for mortality suggests a small yet marginally significant weekend effect for emergency admissions with an odds ratio of 1.04 (95% CI 1.00-1.07).[ii]

Figure 1 Meta-analysis of 11 papers cited as refuting the weekend effect (using risk adjusted odds ratios) [ii]

[i] Assuming a weekday mortality rate for emergency admissions of 5% and a 3:1 ratio of weekday to weekend admissions, with an 80% power to detect a 10% difference in mortality with a type 1 error rate of 5% (1-tailed test).

[ii] RevMan 5 (http://tech.cochrane.org/revman/download) and estimating standard error of Odds Ratio from 95% confidence intervals using Inverse Variance and Random Effects Model (relative risk converted to odds ratio for Mohammed 2016 using Grant, BMJ 2014)

Paul Aylin is Professor of Epidemiology and Public Health, School of Public Health, Imperial College London.

Competing interests: Paul Aylin is co-director of the Dr Foster Unit at Imperial College London. The Dr Foster Unit is partially funded through a research grant by Dr Foster Intelligence (a wholly owned subsidiary of Telstra). The unit also receives funding through NIHR research grants. Paul Aylin is a Non-Executive Director of West London Mental Health Trust.

This blog was unsolicited and peer reviewed.

References:

1. Nasser T, White B, Farne H, A DM, Clarke R, McCartney M, et al. The evidence of an NHS weekend effect is shaky [Letter]. The Guardian; 15/9/2016.
2. Collaboration LS, Virgo C, Abbott BP, Abbott R, Abbott TD, Abernathy MR, et al. Observation of Gravitational Waves from a Binary Black Hole Merger. Physical Review Letters. 2016;116(6):061102.
3. Weber J. Detection and Generation of Gravitational Waves. Physical Review. 1960;117(1):306-13.
4. Astone P, Baggio L, Bassan M, Bignotto M, Bonaldi M, Bonifazi P, et al. IGEC2: A 17-month search for gravitational wave bursts in 2005\2007. Physical Review D. 2010;82(2):022003.
5. Justice for Health. J4H Skeleton Document.
6. Woods CFG, Manohar S, Lindow SW. Obstetric consultant weekend on-call shift patterns have no effect on the management of spontaneous labour in a large maternity hospital. Journal of Obstetrics and Gynaecology. 2013;33(8):802-5.
7. Ferguson HJ, Hall NJ, Bhangu A. A multicentre cohort study assessing day of week effect and outcome from emergency appendicectomy. BMJ Quality & Safety. 2014.
8. Noman A, Ahmed JM, Spyridopoulos I, Bagnall A, Egred M. Mortality outcome of out-of-hours primary percutaneous coronary intervention in the current era. European heart journal. 2012;33(24):3046-53.
9. Rathod KS, Jones DA, Gallagher SM, Bromage DI, Whitbread M, Archbold AR, et al. Out-of-hours primary percutaneous coronary intervention for ST-elevation myocardial infarction is not associated with excess mortality: A study of 3347 patients treated in an integrated cardiac network. BMJ Open. 2013;3(6).
10. Meacock R, Anselmi L, Kristensen SR, Doran T, Sutton M. Higher mortality rates amongst emergency patients admitted to hospital at weekends reflect a lower probability of admission. Journal of health services research & policy. 2016. DOI: 10.1177/1355819616649630
11. McShane P, Draper ES, McKinney PA, McFadzean J, Parslow RC, Paediat Intensive Care Audit N. Effects of Out-of-Hours and Winter Admissions and Number of Patients per Unit on Mortality in Pediatric Intensive Care. Journal of Pediatrics. 2013;163(4):1039.
12. Goldacre MJ, Maisonneuve JJ. Mortality from meningococcal disease by day of the week: English national linked database study. Journal of Public Health. 2013;35(3):413-21.
13. Mikulich O, Callaly E, Bennett K, O’Riordan D, Silke B. The increased mortality associated with a weekend emergency admission is due to increased illness severity and altered case-mix. Acute Med. 2011;10(4):182-7.
14. Smith S, Allan A, Greenlaw N, Finlay S, Isles C. Emergency medical admissions, deaths at weekends and the public holiday effect. Cohort study. Emergency Medicine Journal. 2014;31(1):30-4.
15. Maggs F, Mallet M. Mortality in out-of-hours emergency medical admissions–more than just a weekend effect. J R Coll Physicians Edinb. 2010;40(2):115-8.
16. Li L, Rothwell PM. Biases in detection of apparent “weekend effect” on outcome with administrative coding data: population based study of stroke. BMJ. 2016;353:i2648.
17. Jairath V, Kahan BC, Logan RFA, Hearnshaw SA, Travis SPL, Murphy MF, et al. Mortality From Acute Upper Gastrointestinal Bleeding in the United Kingdom: Does It Display a “Weekend Effect”? American Journal of Gastroenterology. 2011;106(9):1621-8.
18. Mohammed MA, Faisal M, Richardson D, Howes R, Beaston K, Speed K, et al. Adjusting for illness severity shows there is no difference in patient mortality at weekends or weekdays for emergency medical admissions. QJM. 2016. DOI: http://dx.doi.org/10.1093/qjmed/hcw104.
19. Sorita A, Ahmed A, Starr SR, Thompson KM, Reed DA, Prokop L, et al. Off-hour presentation and outcomes in patients with acute myocardial infarction: systematic review and meta-analysis. BMJ 2014;348.
20. Sorita A, Ahmed A, Starr SR, Thompson KM, Reed DA, Dabrh AM, et al. Off-hour presentation and outcomes in patients with acute ischemic stroke: a systematic review and meta-analysis. Eur J Intern Med. 2014;25(4):394-400.
21. Zhou Y, Li W, Herath C, Xia J, Hu B, Song F, et al. Off-Hour Admission and Mortality Risk for 28 Specific Diseases: A Systematic Review and Meta-Analysis of 251 Cohorts. J Am Heart Assoc. 2016;5(3):e003102.
22. Ricciardi R, Nelson J, Roberts PL, Marcello PW, Read TE, Schoetz DJ. Is the presence of medical trainees associated with increased mortality with weekend admission? BMC Medical Education. 2014;14:4.