You don't need to be signed in to read BMJ Group Blogs, but you can register here to receive updates about other BMJ Group products and services via our Group site.


Those unknown unknowns: the importance of publication bias

1 Aug, 09 | by Steven Reid, Evidence-Based Mental Health

Can you believe what you read in a medical journal? Probably not, as many if not most research findings turn out to be false. Poor research design and underpowered studies are part of the problem but looming large in the background is the spectre of publication bias.

No one doubts that negative studies should be published, yet it remains the case that they struggle to get into journals. By negative studies I mean studies that don’t show a statistically or clinically significant effect, or where a new treatment is more effective than standard treatment or placebo but has intolerable or dangerous adverse effects. Much of the blame has been heaped upon those unscrupulous drug companies callously suppressing unfavourable data. But before we all climb up on our collective high horse you should read this month’s Editor’s Choice (free to access) in Evidence–Based Mental Health.

In his personal account of his struggle to publish negative data on the drug lamotrigine, Nassir Ghaemi points the finger at not just the pharma industry, but at the FDA, journal editors and the peer review process itself. A Boston psychiatrist, he speaks as an insider having sat on an advisory board for GlaxoSmithKline as well as the editorial board of the journal Bipolar Disorders (he also writes an entertaining blog, Mood Swings). The contemptuous tone of the rejection letters will be familiar to anyone who’s submitted a paper, as will the contradictory reasons for refusal.

We now have clinical trials registration – requiring that all results end up somewhere in the public domain – which is clearly a good thing. There are also journals such as BMC Research Notes and the Journal of Negative Results in BioMedicine that are specifically aiming to publish negative studies. However the fact that a paper attempting to address publication bias should itself fall victim to that bias indicates that this is a problem that won’t go away.

A drug watchdog shows its fangs…at last

25 Jun, 09 | by Steven Reid, Evidence-Based Mental Health

That drug companies play fast and loose with study data is hardly news. It’s a widely-acknowledged problem that has been highlighted particularly with antidepressant trials where the advantage over placebo is often equivocal. Not before time, someone has decided to draw a line: “Deception through concealment is no trivial offence”, says the director of IQWiG – the German version of NICE – as he accuses Pfizer of concealing data about the antidepressant, reboxetine.

IQWiG (The Institute for Quality and Efficiency in Health Care) has ruled that reboxetine has no proof of benefit. They report that the drug has been tested in at least 16 trials but in 9 of them key information is not reported so they are unable to evaluate the effect of the antidepressant on 3000 of the 4600 patients enrolled. What are the implications of their ruling? The watchdog provides an assessment that is used to inform which medical treatments can be reimbursed through the public health insurance system in Germany – Europe’s largest market for drugs.

A rather sniffy spokesman for Pfizer told the BMJ that IQWiG is not a ‘regulatory authority’ but a ‘private institute’ and that there was no obligation to provide them with information. This seems at odds with Pfizer’s own policy on disclosure: “in all cases study results are reported by Pfizer in an objective, accurate, balanced and complete manner and are reported regardless of the outcome of the study or the country in which the study was conducted.”

Reboxetine itself has had a bit of a mauling of late. The Lancet’s recent comparative meta-analysis of 12 newer antidepressants put reboxetine at the bottom of the pile both in terms of efficacy and tolerability. It’s also not available in the US as the FDA turned down the application for a license for reasons which are still unclear. Paradoxically, if this license had been granted current US legislation would have required full disclosure of all the trial data. IQWiG is now calling for a similar European Union-wide legal obligation to publish all trial results as clearly self-regulation by the industry is not working. Corrado Barbui, writing in EBMH, made the same point back in 2007. Isn’t it time to get on with it?

Antiepileptic Alert: scaremongering or suicide prevention?

8 Jan, 09 | by Steven Reid, Evidence-Based Mental Health

The debate has been rumbling on for a year, but last month the US Food and Drug Administration (FDA) announced that manufacturers of antiepileptic drugs will have to add warnings to package inserts stating that these drugs increase the risk of suicide. They have also issued a public health advisory requiring that “health care professionals should notify patients, their families and caregivers of the potential for an increase in the risk of suicidal thoughts or behaviors so that patients may be closely observed”. These alerts arrive following a review of 199 clinical trials that showed a doubling of risk of suicidality (not completed suicide) in people using antiepileptics. So warning patients makes sense doesn’t it? But then why are so many neurologists, psychiatrists and notably the American Epilepsy Society unhappy with the decision?

It boils down to the perception of risk and the old relative vs. absolute question. The trials included over 40000 patients and found that the absolute risk of suicidal thinking or behavior was 0.43% in the group taking antiepileptics versus 0.24% in the group on placebo. That equates to one additional case of suicidality (again, not completed suicide) for every 500 patients taking the active drug. Do antiepileptics seem quite so scary now?

The FDA says that the warnings are needed to promote monitoring for a significant, even if small, risk. Given that epilepsy itself is associated with a threefold increased risk of completed suicide watching for signs of depression should be routine. As well as epilepsy antiepileptics are also used in bipolar disorder and with chronic pain – conditions with their own significant risk of suicide. But a regulator urging increased monitoring doesn’t seem to affect clinical practice. Take the case of antidepressant prescribing in adolescents (see an earlier post). When trials indicated an increased risk of suicidality with SSRIs the FDA slapped on a black box warning. The adolescent suicide rate went up (possibly coincidental) and prescribing rates plummeted (probably not coincidental). That’s the concern with this latest alert: people will be put off taking antiepileptic drugs, and unlike antidepressants there is no alternative treatment. Providing information is important but so is context and I think this is a decidedly dubious decision by the FDA.

EBMH blog homepage


A quarterly digest of the most important clinical research of relevance to clinicians in mental health.Visit site

Latest from Evidence-Based Mental Health

Latest from Evidence Based Mental Health