Recently, Benjamin Leis (@LeisBenjamin) and colleagues published a short report on an intervention in a cardiac intensive care unit (ICU) in Canada. In this study, the authors removed TSH testing from standard admission order sets with the aim of reducing the number of unnecessary thyroid studies obtained.

Often, altering an order set like this is the end of the story:  an electronic health record (EHR) change is implemented, and the result eyeballed to check it seems to work. In this case, however, the authors performed a pre-/post-intervention time-matched analysis, comparing ordering patterns in their cardiac unit with that of a neighboring, non-cardiac control ICU. This control unit ordered roughly 1/8^th the number of thyroid studies before the order set change was put in place.

Overall, the authors found a drop in overall testing rates in their unit, with no change in ordering rates within the comparison ICU. So far, so good. Additionally, however, they made two further, rather less welcome discoveries: 1) up to 1/3 of patients NOT tested for thyroid function might actually have had an appropriate indication for the test (so might have benefited from having it done, but did not), and 2) the number of thyroid tests obtained in the post-intervention period were more likely to have been drawn without a physician order than with a physician order. This meant that after the admission order set eliminated automatic testing for thyroid studies, staff in the unit often continued routinely drawing and sending the tests even though the physicians had not requested them.

The first unexpected finding illustrates a potential downside of relatively inflexible EHR order sets. As described within the report, previous work has suggested that order sets might lead physicians to change critical thinking behaviors, relying on what is presented rather than seeing it as a starting point to add/subtract additional orders based on the immediate clinical scenario.¹ Though several studies have concluded that removing automatically scheduled lab tests from order sets can reduce overuse, not all use is inappropriate. Some testing is important to clinical care.^2,3 In this case, adverse clinical outcomes related to eliminating automatic thyroid testing were not identified by the authors. But it’s not hard to imagine that removing tests or medications from order sets in other situations might lead to negative consequences for patients. Striking the balance between appropriate and inappropriate testing and ordering therefore requires a fully engaged clinician both aided and unimpeded by the electronic tools available. Projects aimed at reducing lab testing often will need to deliberately look for balancing measures related to unintended patient safety outcomes.

The second unexpected finding provides insight into organizational memory within clinical units. In this study, thyroid testing endured as a practice, absent of explicit orders by the clinicians. One possibility is that ordering of thyroid tests might, during the pre-intervention period, have been driven by efficiency; these labs were obtained during the majority of admissions, so a lack of a TSH order might have been perceived by staff as a simple oversight and therefore was done to streamline workflow. In the post-intervention period, staff memory of the previous order set that had requested TSH testing may have persisted long after it was no longer intended.

One reason for favoring EHR changes for lab ordering is to create stronger interventions that avoid the known effects of popular, but often ineffectual, educational interventions. In this case, orders for tests by clinicians decreased dramatically. However, many patients continued to receive thyroid testing without a clinician order. Over time, the collective memory of required TSH testing will likely fade in favor of one with fewer thyroid studies. However, this change will probably take longer than suggested by the authors’ original eyeball test, suggesting that implementing even small and straightforward EHR changes often will require evaluation of multiple workflows for unintended consequences.

B Leis et al. Altering standard admission order sets to promote clinical laboratory stewardship: a cohort quality improvement study.

If you have had similar experiences with what you thought was a simple change not going as planned, please share in the comments below!

¹Ash JS, Sittig DF, Poon EG, et al. The extent and importance of unintended consequences related to computerized provider order entry. Journal of the American Medical Informatics Association. 2007;14: 415-23.

²Eaton KP, Levy K, Soong C, et al. Evidence-based guidelines to eliminate repetitive laboratory testing. JAMA Intern Med. 2017;177(12):1833-1839.

³Korenstein D, Husain S, Gennarelli RL, et al. Impact of clinical specialty on attitudes regarding overuse of inpatient laboratory testing. J Hosp Med. 2018;13(12):844-847.