Most scientific studies are wrong, and they are wrong because scientists are interested in funding and careers rather than truth.

That was the chilling message delivered by the smiling, brilliant, erudite, and cuddly John Ioannidis at the Seventh Peer Review Congress in Chicago this week. Listening to somebody as brilliant as Ioannidis is like listening to a great opera or watching a gripping football match: you feel inspired, uplifted, and privileged. And, although I would never describe a female speaker as cuddly (no matter how cuddly she might be), I write this about Ioannidis because it felt good to see such brilliance worn so lightly and attractively.

His report  “Why most published research findings are false” is the most cited paper in PLOS Medicine and has contributed to him being profiled in the New York Times and becoming famous.

Beginning his talk with a poem from Sappho, the Ancient Greek poet, he showed us a picture of the fragments of parchment each containing a few squiggles from which the poem has been constructed. The relationship between our scientific knowledge and the data from which it has been constructed is equally fragile.

Ioannidis illustrated his theme by describing a study in which colleagues randomly selected 50 ingredients from the Boston Cookbook and then searched PubMed to see which of the ingredients had been linked with either increasing or decreasing the risk of cancer. The answer was 40. Large numbers of studies had shown so, and no doubt hundreds of thousands of media reports will have spread the message to the public. In fact, said Ioannidis, a meta-meta-analysis shows that the scientific studies are “correct” in almost no cases. (The public may be smarter than the scientists in discounting and ignoring these reports, although an unfortunate result is a public scepticism about science that leads many to accept the assertion that there is no evidence for human activity causing climate change.)

A huge growth area in science is linking genes with particular diseases, and Ioannidis presented an analysis of hundreds of such studies showing that in only 1.1% of cases was the linkage true. Think of that when you next hear on the radio that a gene has been found for depression, schizophrenia, or obesity. The reports suggest that a “cure” is closer, but in fact there is probably no true linkage at all.

Biomarkers of diseases are another subject where we are being constantly misled, with highly cited studies showing a high relative risk almost always turning out to have greatly exaggerated the relative risk. The same is true for prediction models for disease, and a study of 127 models showed that in only a fifth had there been further studies to validate the model. In half of that fifth the further studies came from the original authors, which is unsatisfactory.

Companies and hedge funds are catching up with the fact that so many published studies are misleading—because it’s one thing to have the study published in a journal, with the publishers making money and the authors enjoying “fame and the love of beautiful women,” but it’s another thing to invest millions of dollars in what appear to be new possible treatments or diagnostic tests when the result may be wrong. So companies are learning the importance of replicating studies, and a recent study by Amgen of preclinical studies showed that 80-90% could not be replicated. Hedge funds have thus become nervous about investing in what seem to be promising scientific results and are hiring contract research organisations to replicate studies before they make any investments.

Almost as an aside Ioannidis showed the results of which countries produce the highest proportions of misleading studies. The US is at the top, perhaps because the pressure to publish is greater there than anywhere.

Why, asked Ioannidis, are so many scientific studies wrong? Because of bias and random error, and there is “plenty of both.” An analysis of bias in 17 million papers showed 235 sources of bias, most of which, said Ioannidis, were not familiar to him, a student of bias. He has an ambition to publish the “Definitive Encyclopedia of Bias.”

Random error is so important because most of modern epidemiology and health is about small effects and weak associations. Odds ratios of over five are seen in only one in 16 studies, almost always in small studies, and with 99% of those studies the effect size shrinks when a larger study is done. A study of 85 000 meta-analyses showed that only one had a big effect that was highly significantly. “We have to learn to live with small effects,” concluded Ioannidis.

Another “solution” to the problem of so many misleading studies is to encourage large collaborations that can generate enough data to detect reliably small effects. Improved reporting and registration of studies are also needed. Ioannidis showed that most important journals now have requirements around registration, but in most cases it isn’t actually happening. We have the policies but not the reality, something that is seen again and again with scientific journals.

But the most important need is to reproduce studies, and several initiatives are underway to make this happen. For example, the recently launched Reproducibility Initiative encourages people to submit their studies for independent validation.

Why, asked Ioannidis, at the end of his talk are we doing science? Contentment with a system that encourages the publication of studies that are mostly misleading suggests that it’s about careers, grants, publications, and salaries. If it’s about a search for “truth” then we need more collaboration, less publishing of small and biased studies, and a heavy emphasis on reproducibility.

Richard Smith was the editor of the BMJ until 2004 and is director of the United Health Group’s chronic disease initiative.

Competing interest: RS was given free admission to the congress and had his expenses paid by JAMA because he is presenting a film on the history of evidence based medicine and was master of ceremonies at a “roast” of Drummond Rennie, the creator of the peer review congresses and a great friend of RS.