Outcome reporting bias: cherry-picking the best results

When planning an RCT, the choice of primary outcome is crucial. This is an integral part of the research question, and forms the basis of the sample size calculation. Secondary outcomes are also chosen, to give a wider indication of the effects of interventions, generate new hypotheses, and contribute to meta-analyses.

In many trials, outcomes are selectively reported, on the basis of the results, or the reported primary outcome differs from the one specified at the trial outset (again, often changed on the basis of the results). These practices lead to outcome reporting bias.

Selective outcome reporting renders the study report a biased reflection of the overall trial findings, which is misleading for the reader, and has substantial impact on the conclusions of Cochrane reviews (whose meta-analysis comprises positive results, but not all the negative results).

It can be difficult to evaluate for outcome reporting bias in an individual trial. It is important to check that all outcomes described in the methods are subsequently accounted for in the results. Consider whether other outcomes you might have expected to see are missing. It is also useful to check the trial protocol (if available) to check for outcomes that were measured, and not reported. In some conditions, core outcome sets have been agreed (which should be measured and reported in all trials in that condition), and it may be useful to compare these with what was reported in the trial.