{"id":1332,"date":"2016-09-20T21:53:55","date_gmt":"2016-09-20T20:53:55","guid":{"rendered":"https:\/\/blogs.bmj.com\/adc\/?p=1332"},"modified":"2017-10-30T13:42:05","modified_gmt":"2017-10-30T12:42:05","slug":"but-if-its-significant-it-must-be-true","status":"publish","type":"post","link":"https:\/\/blogs.bmj.com\/adc\/2016\/09\/20\/but-if-its-significant-it-must-be-true\/","title":{"rendered":"But if it’s significant it must be true?"},"content":{"rendered":"

One thing that I keep coming across, from a huge range of folks involved in clinical practice, is the idea that if something is statistically significant, then it’s true.\u00a0Some folk nuance that a bit, and say things like “true 95% of the time” for confidence intervals or p=0.05 …<\/p>\n

Of course, there’s an ongoing argument about exactly how to understand p-values in common, understandable language. A simplish and we hope right-enough version can be found here<\/a>. But underlying that is a different, more important truth.<\/p>\n

The stats tests work to\u00a0assess what might be the product of chance variation. When\u00a0the data they are testing come from hugely biased studies, with enormous flaws, and\u00a0the poor little stats machine says p=0.001, the researcher and reader may conclude “this is true””. This is wrong: it is due to bias and poor research.<\/p>\n

It may be better to think “this is unlikely to be due to chance” – in remembering that phrase, you’ll hopefully recollect the other reasons why something may not be due to chance too.<\/p>\n

 <\/p>\n