In the last post I discussed the ‘p’ problem (not enuresis, which is subject to an upcoming NICE guideline) but statistical significance is only the first problem in deciding if something actually works. This post takes up the challenge of not just saying that something is likely to work, but just how well it works. This is the realm of the comparative effectiveness measures – relative risk, odds ratio, risk difference and ‘number needed to treat’ – and the confidence interval… which will come later.

Take a treatment for croup (say steroid) and compare it to placebo. Let’s say 10/20  of the steroid group go home from the ED, and 5/20 of the placebo group do. The absolute benefit of steroid is 5 fewer hospitalisations for each 20 patients, which is the same as 1 patient for every 4 treated: the number needed to treat is 4. This can also be expressed as a risk difference (RD) of 0.25. (This is the absolute difference in proportions: 0.5 – 0.25 = 0.25.) The arithmetic among you will have noticed that the inverse of 0.25 (1/0.25) is 4 – this is a quick route to the NNT. These are absolute measures of effectiveness – they say how many people, or what proportion of a population, will benefit from your treatment over and above the ones who would have got better anyway.

Now look at it a different way. The steroid has taken the chance of being sent home up from 25% to 50% – it has doubled each patient’s ‘risk’ of being sent home. Or, the ‘relative risk’ (RR) of going home is 2.  This is a relative measure of effectiveness – it says what the changing chance of an outcome is for each individual patient, family or the like. The odds ratio can be interpreted, in a clinical sense, the same way as a relative risk.

So what’s the use of the two ways of thinking about the same stuff? Well, we can make more rational decisions about the balance of risk and benefits with absolute numbers (1 in 55 survive, but 1 in 14 go deaf…)  and extend this to whole communities (can we afford to spend £30,000 per patient for 1 in 121 to benefit?). Relative measures can be ‘transported’ across different settings with some ease … if the RR is 2 when the placebo send-home rate is 25%, it’s also likely to be 2 if the rate is 12.5% … if this were the case the RD would be 0.125 and the NNT would then be 8. The other thing that RR do is ‘sell’ something: what’s better, the treatment that works in only 1 in 4 people or the one that doubles the chance of success?

So the simple measures of success are those for communities and balances – the absolute ones – and the personal, transportable and marketing  – the relative ones. Recognise them, use them, and you’ll be lots further on in understanding rational decision making.