Simon Wessely: The lost trial—a Christmas story

simon_wesselyA long long time ago, well, 1995 if you are asking, I was part of a team running a randomised controlled trial. I was, and indeed still am, working in a general hospital as a liaison psychiatrist. Back then, and still now, a considerable proportion of those admitted to the general hospital wards had problems with alcohol misuse. And back then, and still now, it was not clear what should be done about it. An Edinburgh team led by psychiatrist Jonathan Chick had published a paper showing that screening for alcohol related problems followed by a brief nurse led counselling intervention had a positive impact on the frequency of substance misuse. [1] However, one swallow does not make a summer, even when published in the BMJ, and it seemed important that the trial was replicated and extended.

And so we conducted just such a trial at King’s College Hospital with the support of a grant from the now defunct South East London Health Authority.  The main investigators were Jane Marshall (JM), already a well known addictions psychiatrist, and Tim Peters (TP), our professor of biochemistry with a major reputation in alcohol related disorders. Of course the bulk of the work was done, as ever, by the project team, ably led by Una Canning (UC). A screening programme was started, 254 people with at risk drinking were identified, and 154 were randomised. The trial was completed on time, and to budget. A manuscript was prepared and submitted to the BMJ. The paper was rejected, albeit with encouraging comments from the referees.

And now the problems started.

By this time the research team had split up. The grant was finished, so the project coordinator who had prepared the first draft (UC), moved to another post outside research, as did the two research assistants. JM started to revise the manuscript following the suggestions of the BMJ referees. However, JM moved into a full time clinical post, and was exceptionally committed to new service developments. Myself and TP meanwhile were also involved in new academic projects, while I was also very busy setting up two new academic units. Contact was lost with UC. I tried a few times to resurrect the trial but was hampered by only having an out of date paper version of the manuscript and no further access to the data set. Gradually the memory of the trial started to fade away.

And then I went to the 2011 BMJ Christmas party.  And there I met Iain Chalmers, the terrifying God of all things Cochrane, and Doug Altman, doyen of statisticians and a longstanding stalwart of the journal.  The topic turned to the issue of unpublished trials, which even before Ben Goldacre launched the All Trials campaign, was already being much discussed. I found myself confessing that I had actually been part of a group that had never published a trial that had concluded over a decade ago.  No, it wasn’t a Bad Pharma plot, indeed it was nothing whatsoever to do with pharma, but a trial done to a reasonable standard, in a field that remained very active and relevant to current health policy. I was naturally berated by Iain, and prepared to hang my head in shame.  And then Doug Altman mentioned a new journal, with the catchy name of Trials, that was committed to publication of trials, even old ones.

And so I decided to have another go at getting the trial to publication. Although we were unable to locate the data set, JM managed after a considerable effort to find the electronic copy of the manuscript, which included the revisions suggested by the BMJ and our responses. Neither of us, however, had the time (a familiar lament) to get the old manuscript into a presentable condition, nor to update the literature which had continued to expand in the intervening years, and now included several meta-analyses and a Cochrane review, albeit none of them including our trial, given that the authors did not have the gift of clairvoyance. And so I suggested we propose a Special Study Component (SSC) for a King’s College London medical student, consisting of a project on non publication of trials, linked to resurrecting and updating an example of the genre. And so we now had a new member of the team, medical student Celia Shiles (CS). She got to work with gusto, updated the introduction and discussion, and carried out a meta-analysis to see how our trial would influence knowledge. And so 19 years after we wrote the grant, and 17 years after we finished the study, we submitted the trial to Trials.

Of course the trial, or more specifically, the statistics, were showing their age. And like most trials it was underpowered, even though we had randomised over 150 people. We made the deficiencies in the trial clear, the manuscript was accepted, and our trial is now published for all to see. [2]

We feel that there is a message in this.  It is easy to claim that non-publication of trials may be result of deliberate decisions, such as a wish not to publicise negative results, and we know this does indeed happen.  Part of the All Trials campaign has been to propose various rather unpleasant penalties that should be visited on those guilty of failing to publish a clinical trial within a reasonable period time. Opinion differs as to what is “reasonable,” but no one has been suggesting the 14 year period that had already elapsed since we finished our trial. Clearly the prison gates would have long since closed around us  But probably more common is a more understandable and human sequence of events. A first attempt at publication fails. Teams break up. Some leave academic life and start families.  Others move into busy clinical posts, or take on new demanding projects. And time marches on.

But what we hope we also show is that all is not lost. The most important thing is that we have finally got our principal results into the public domain. Of course this is not ideal, but perhaps the most important thing is that the data from this trial are now available to all, and in a form that enables them to contribute to the next systematic review and meta-analysis in a subject that continues to be a very important public health problem, and for which we are still short of definitive RCT evidence
So we feel that even after over a decade, this study makes an important contribution to an ongoing debate and should at least be available to readers and to the authors of future systematic reviews and/or meta-analyses. We hope that others will now be persuaded to look into their own file drawers and see what lies there.  It really is never too late. [3]

And most important of all—next week is the BMJ Christmas party.  And this time I can look Iain and Doug in the eye again.

Simon Wessely is a professor, chair and head of the department of psychological medicine, Institute of Psychiatry, King’s College London.

1.    Chick J, Lloyd G, Crombie, E. Counselling problem drinkers in medical wards: a controlled study. BMJ 1985; 290: 965-967.

2.    Shiles C, Canning U, Kennell-Webb S, Gunstone C, Marshall J, Peters T, Wessely S. Randomised controlled trial of a brief alcohol intervention in general hospital setting Trials 2013: 14; 345

3.    Doshi P, Dickersin K, Healy D, Vedula S, Jefferson T: Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ 2013;346:f2865