How you know about stuff you don’t know about is a fascinating philosophical, statistical and downright practical topic.

There’s the issue of ‘unknown unknowns’ – stuff you didn’t know you didn’t know – that can be frighteningly and sometimes dangerous. (How may of you know that failure to thrive is a common manifestation of a midline low grade brain tumour?). Then there’s the issue of missing information from trials – failure to collect it can lead to underpowered estimates and an inability to get to an answer and wasting the effort of all the patients. And there’s the issue of suppressed, unpublished or fraudulent stuff – but more of that in a different blog post.

Beyond that is just stuff that hasn’t happened yet – for example, if we give 100 children on immunosupressive therapy probiotic yoghurts and see a reduction of diarrhoea, and no cases of infection, are we happy that the yoghurt is giving some tummy loving cuddles and isn’t dangerous? How many pots of the white stuff do we need to see eaten before we stop worrying about sepsis?

Well, we can estimate this in a few ways. The simplest is using the 3/n rule (if you’ve not seen it happen, then there’s still a 3/n – 3/100 chance in out example – that it will happen with 95% confidence). More sophisticated is the formal estimation of a binomial confidence interval (again, more can follow about this if we get any comments requesting it via the blog or twitter!). But the idea is the same – even when we don’t know, we can use what we do know to make an educated guess at it.

Remember this when faced with new treatments, pathognmonic signs or prognotic certainties. You can know what you don’t.