Can you believe what you read in a medical journal? Probably not, as many if not most research findings turn out to be false. Poor research design and underpowered studies are part of the problem but looming large in the background is the spectre of publication bias.

No one doubts that negative studies should be published, yet it remains the case that they struggle to get into journals. By negative studies I mean studies that don’t show a statistically or clinically significant effect, or where a new treatment is more effective than standard treatment or placebo but has intolerable or dangerous adverse effects. Much of the blame has been heaped upon those unscrupulous drug companies callously suppressing unfavourable data. But before we all climb up on our collective high horse you should read this month’s Editor’s Choice (free to access) in Evidence–Based Mental Health.

In his personal account of his struggle to publish negative data on the drug lamotrigine, Nassir Ghaemi points the finger at not just the pharma industry, but at the FDA, journal editors and the peer review process itself. A Boston psychiatrist, he speaks as an insider having sat on an advisory board for GlaxoSmithKline as well as the editorial board of the journal Bipolar Disorders (he also writes an entertaining blog, Mood Swings). The contemptuous tone of the rejection letters will be familiar to anyone who’s submitted a paper, as will the contradictory reasons for refusal.

We now have clinical trials registration – requiring that all results end up somewhere in the public domain - which is clearly a good thing. There are also journals such as BMC Research Notes and the Journal of Negative Results in BioMedicine that are specifically aiming to publish negative studies. However the fact that a paper attempting to address publication bias should itself fall victim to that bias indicates that this is a problem that won’t go away.

That drug companies play fast and loose with study data is hardly news. It’s a widely-acknowledged problem that has been highlighted particularly with antidepressant trials where the advantage over placebo is often equivocal. Not before time, someone has decided to draw a line: “Deception through concealment is no trivial offence”, says the director of IQWiG – the German version of NICE – as he accuses Pfizer of concealing data about the antidepressant, reboxetine.

IQWiG (The Institute for Quality and Efficiency in Health Care) has ruled that reboxetine has no proof of benefit. They report that the drug has been tested in at least 16 trials but in 9 of them key information is not reported so they are unable to evaluate the effect of the antidepressant on 3000 of the 4600 patients enrolled. What are the implications of their ruling? The watchdog provides an assessment that is used to inform which medical treatments can be reimbursed through the public health insurance system in Germany – Europe’s largest market for drugs.

A rather sniffy spokesman for Pfizer told the BMJ that IQWiG is not a ‘regulatory authority’ but a ‘private institute’ and that there was no obligation to provide them with information. This seems at odds with Pfizer’s own policy on disclosure: “in all cases study results are reported by Pfizer in an objective, accurate, balanced and complete manner and are reported regardless of the outcome of the study or the country in which the study was conducted.”

Reboxetine itself has had a bit of a mauling of late. The Lancet’s recent comparative meta-analysis of 12 newer antidepressants put reboxetine at the bottom of the pile both in terms of efficacy and tolerability. It’s also not available in the US as the FDA turned down the application for a license for reasons which are still unclear. Paradoxically, if this license had been granted current US legislation would have required full disclosure of all the trial data. IQWiG is now calling for a similar European Union-wide legal obligation to publish all trial results as clearly self-regulation by the industry is not working. Corrado Barbui, writing in EBMH, made the same point back in 2007. Isn’t it time to get on with it?

Trying to find evidence to guide practice in mental health can be perplexing at the best of times. Take antidepressants for example: seemingly prescribed by the bucketload (170 million prescriptions issued per year in the US, 31 million in the UK) many GPs and psychiatrists will admit that when it comes to choice of drug, more often than not it’s a case of ‘suck it and see’. We may have a raft of sophisticated neuroimaging techniques but we still struggle to explain how they work and when it comes to predicting response – well that’s anybody’s guess.

These days, we view pharma-funded trials with a jaundiced eye, particularly when they are promoted by ‘key opinion leaders’ (to see one KOI meet his nemesis, take a look at the Carlat Psychiatry Blog), so I would recommend this month’s Editor’s Choice (that means it’s free to access) in Evidence-Based Mental Health. Simon Hatcher casts a critical gaze over the STAR*D trial, the largest study of treatments for depression ever undertaken. It is one of the new breed of pragmatic trials – studies that assign ‘real world’ patients to licensed drugs aiming to assess their effectiveness (do they work in real life?) rather than efficacy (do they work in ideal conditions?). Publicly funded, at a cost of $35 million, and enrolling almost 3000 participants STAR*D really is, as Hatcher states, the 300lb gorilla of antidepressant trials. So what does STAR*D tell us? Well, read the review to find out but it’s clear that quality is just as important as quantity.