Assessing the effects of health policy interventions on patient care and outcomes is essential for improving their design and implementation. The ideal health policy is implemented in a manner that permits its prospective evaluation. In reality, however, it is seldom the case, and understanding their effects often requires analyses that disentangle their effects in real world data, while accounting for concurrent changes in the healthcare landscape.

A program that was implemented under the Affordable Care Act in the US has received significant attention. This program—the Hospital Readmissions Reduction Program (HRRP)—was aimed at reducing readmissions through financial penalties on hospitals with higher than expected readmissions. There has been a deluge of information in scientific journals and lay media addressing the effects of the program. These have particularly looked at whether the program was associated with an increase in patient mortality, especially in heart failure. In the period spanning the introduction of the HRRP, Medicare beneficiaries hospitalized with heart failure more frequently survived to hospital discharge, but experienced increasing mortality in the 30-day period following hospital discharge. The temporal association with HRRP’s introduction varies by the methodological approach used. However, the critical gap in our knowledge is whether hospitals pursued a reduction in readmissions by avoiding necessary readmissions, which resulted in increased mortality. 

The key to evaluating such policies should be to employ rigorous science, and testing hypotheses and assumptions in ways that are rigorous and transparent. Moreover, there should be a conceptual model to illuminate a mechanism. In that spirit, we conducted a series of data experiments, evaluating patterns of hospital discharges, readmissions, and patient mortality.

First, if hospitals were reducing readmissions by pursuing strategies that increased the mortality risk, then hospitals that reduced readmissions should have an increase in mortality. However, a study showed that hospitals with a reduction in readmission tended to have a reduction in mortality.

Second, if hospitals were configuring their care in 30 days to reduce access to hospitalization, then there would be an expected increase in hospitalizations starting at day 31 when the policy was no longer operative. However, in an evaluation of this issue, there was no evidence for deferral of readmissions at the 30-day threshold.

Finally, if hospitals were denying patients admission who were coming to the hospital for acute care, then we would have expected those patients seen in the emergency department or in observation wards, but not hospitalized to have the increase in mortality. However, the patients seen in acute care settings, but not admitted did not have an increase in mortality. Mortality increased only in patients who were neither readmitted nor sought care in the emergency department or observation units, with nearly half of deaths in 30-day post-discharge period in this group among those discharged to hospice. This is an important observation, especially as hospice discharges are an imperfect surrogate for end-of-life care when US adults predominantly die at home. While the study cannot definitively conclude that appropriate triage of in-hospital deaths to the post-discharge period as a part of end-of-life care underlies the mortality trend, it provides an alternative explanation worthy of dedicated investigation. 

Collectively, these mechanistic, hypothesis-driven assessments can aid the scientific discussion of the effects of the HRRP. In the future, randomized implementation and evaluation of policies, such as the HRRP, may help infer causal associations. However, even for drugs and devices that are mostly approved following randomized trials, real-world assessments help evaluate the many ways in which care and patient outcomes are affected. 

Therefore, generating mechanistic hypotheses about the effects of health policies and testing them with carefully designed experiments in real-world data will remain essential even where novel strategies to implement policies are explored.