Subgroups and multiple analysis. Truth or herrings?

river_herring_2We recently published, in the F&N edition, a paper reporting an RCT looking at inhaled steroid in wee premature babies to see if the treatment reduced the incidence of death and chronic lung disease.

Did we do a good job?

The trial was prospectively registered, before enrolling patients, randomised and stated it was assessing (on the registry) the primary outcome of:

Death or use home oxygen therapy (HOT) for chronic lung disease

and secondary outcomes of

1. Death

2. Death or Incidence of chronic lung disease at 28 days

3. Death or Incidence of chronic lung disease at 36 weeks of postmenstrual age

4. Neurological and developmental outcome at 1 and half year old

5. Neurological and developmental outcome at 3 years of age

The paper reported

“The primary outcome measure used to indicate the morbidity of severe BPD was death or oxygen dependence at discharge. The secondary outcomes were death, severe BPD and neurodevelopmental outcomes at 18 months of PMA and 3 years of age.”

So – we may have failed to identify in our editorial processes that the secondary outcomes #2 and #3 weren’t reported, and the occurrence of severe BPD slipped into outcomes #4 and #5. I’m OK with the primary outcome – after all, oxygen at discharge == home oxygen therapy sort of.

The paper also reports that

“we evaluated subgroups divided by GW at birth (22–23, 24–26 and over 26), CAM [chorioamnionitis] and RDS. The p values for the subgroup analyses are p values for interaction. We did not control for multiplicity because these analyses were exploratory”

Now, I have an instinctive dislike for subgroup analysis to find The Truth about stuff – there’s an entire blog on it here if you like star signs and such – but this seems like a very up front description. We did this as a fishing trip – see what popped up – what might be worth looking at. Great!

But …

The discussion mentions

“we demonstrated that FP [fluticosone] treatment significantly decreased the risk of death or oxygen dependence at discharge in the subgroup of infants born at 24–26 GWs and in infants with CAM . Subgroup analyses also showed the benefits of inhaled steroids for infants born at 24–26 GWs with RDS.”

… not a lot of exploratory emphasis in that chunk … and the abstract’s results include

” In subgroup analyses, the frequencies of death and oxygen dependence at discharge were significantly decreased in the FP group for infants born at 24–26 GWs and for infants with CAM, regardless of the GW at birth.”

.. though to be fair (assuming that gestational age and extremely low birth weight are fairly intimately linked) the conclusion is

“Inhaled steroids have no effect on the prevention of severe BPD or long-term NDI [neurodevelopment] but might decrease the severity of BPD for ELBWIs with a risk factor.”

Hmmm.

How did we do with this – in terms of reporting all outcomes and sticking with the registered study protocol – not great. There may be a chance that this is over-interpreted by some readers as demonstrating ‘the truth’ of what an exploratory analysis showed. We desperately DO want to publish RCTs in children; unpublished studies are unethical and unhelpful. We do want to ensure that our published reports are as truthful and meaningful as possible.

But as this blog KEEPS ON GOING ON ABOUT you shouldn’t read anything on face value and always appraise … even if it’s from us.

  • Archi

(Visited 105 times, 1 visits today)