The last few years have seen clarity in the emergence of an insidious and tainting perversion of the truth through data obfuscation, suppression and misalignment. The challenge of unseen trials has been addressed in a number of ways; probably the best of these are the insistence on all trials in human subjects being prospectively registered with their protocols on public view, twinned with the current campaign to make ALLTRIALS public. There’s a further meta-analytic approach which is a bit geekier, a bit less reliant on finding the hidden, and more focussed on using what we already know to guide us to what we don’t.

The core of this idea sits on the a funnel plot; this is a graphical representation of magnitude of effect against precision of the trial estimate (which is mainly derived from the inverse of the trial size – bigger trials are more precise).

Asymmetry in a funnel plot can be from a number of reasons, including true heterogeneity (different effects in different subgroups), chance, publication or reporting biases. In those settings where true heterogeneity appears unlikely, regression techniques have been used to estimate what the unbiased effect is.

Put simply, we’re using the same techniques that link shoe size, stride length and height, but looking at the relationship between trial ‘size’ and magnitude of effect. If we can predict that relationship, then for the biggest imaginable trial in the world ever we can say what it’s effect size would be – that’s the best guess at the true effect of a treatment.

So if you see a meta-analysis with a funnel plot suggestive of small study effects / publication bias, you might also see the use of a regression based analysis to come up with an adjusted, more accurate effect. Embrace your inner geek and go with it.

– Archi