Recently, perceived shortcomings of The BMJ peer review system have been extensively discussed on Twitter and elsewhere because we rejected a research paper by Rachel Meacock and colleagues examining the “weekend effect.” The paper was ultimately published in The Journal of Health Services Research and Policy. An author of the paper took the unusual step of publicly identifying Professor Nicholas Freemantle as one of The BMJ peer reviewers and implied that he had inappropriately declared no conflict of interest related to the paper under review. Professor Freemantle’s identity was known to her because of The BMJ’s open peer review process.
Feelings ran high, since all of this occurred in the midst of the rancorous junior doctors’ dispute. Some assumed that the paper was rejected solely because of Freemantle’s review, which was not the case. Some also felt that Freemantle’s previous publication of a paper on the same topic, which reached a different conclusion, constituted a conflict of interest that should have been reported or disqualified him as a reviewer.
Ordinarily, The BMJ does not comment on submitted papers until and unless we publish them. We also assume that, although authors will know who the peer reviewers were, their identity and comments will remain confidential unless we publish the paper in The BMJ. In the case of the Meacock paper, however, we are going to break with precedent. We do this with the permission of the authors and the reviewers, in order to clarify our expectations about the behaviour of authors and peer reviewers, and explain how the decision about this paper was made.
Open peer review at The BMJ
We believe that open review promotes constructive conversation while still providing comments that help authors improve their manuscript and editors make a decision about publication. The system gives credit to reviewers while making them accountable for their comments. Reviewers for The BMJ agree to have their signed comments posted if a paper is published, but not otherwise.
One argument against open peer review has always been the potential for reprisals against reviewers. Since we instituted open review in 1999 we have heard of few instances of retaliation. This incident is one of them, and we are grateful to Professor Freemantle for his resilience and professionalism in the face of the unfounded criticism he has received on social media. We continue to think that open peer review is a more responsible system than any alternative. Like all participatory approaches to decision making, though, it relies on the goodwill and cooperation of those who take part.
Our instructions to reviewers make it clear that papers under review are confidential and should never be disclosed to anybody else. We realise that our instructions to authors have not been similarly explicit about confidentiality and the considerate behaviour we expect towards reviewers. We will revise them to clarify these matters. We encourage authors to contact us directly if they have concerns about the fairness of the process.
How the decision was taken not to publish the Meacock paper
The idea of an all-powerful reviewer who can control what we publish is impossible to reconcile with the way our process actually works. During our weekly manuscript meeting, a team of research editors and a consulting statistician decide which papers to publish. We take reviewer comments into account, but also consider how relevant the paper will be for our readers, the quality of the study methods, its novelty and timeliness, and how it compares to other papers we have to consider.
The decision to reject the Meacock paper was made by The BMJ editors and not by the four external peer reviewers who saw the original paper and a subsequent revision. With permission from the authors and reviewers we have posted their signed comments, which can be viewed here along with editorial comments. In agreeing to our request three of the reviewers, including Professor Freemantle, told us, unprompted, that they had hoped we would publish the paper and were surprised that we didn’t. We suspect that many people who read the reviewer comments may also think we made a mistake.
We ask our peer reviewers to declare all their potential conflicts of interest, financial and otherwise. Professor Freemantle tells us that he did not believe that having published a paper in an area represents such a conflict and that his position on the topic was well known to all involved. We agree with those who suggest that he might have included in his competing interests declaration a statement that he had published a paper on the topic in question that reached a different conclusion. But this made no material difference to the review process; indeed, he was selected precisely for this expertise because we seek opinions from people who are active in the relevant field.
Almost everyone who is invited to be a peer reviewer has published papers on related topics, and frequently they have reached different conclusions from the authors whose paper they review. We do not think this disqualifies someone as a reviewer. In fact, such reviews help us identify the criticism a paper might face if it is published.
In summary, The BMJ research editors assume responsibility for rejecting the Meacock paper. It is inappropriate to blame the peer reviewers for our decision, and certainly wrong to put the blame onto one of them alone. It is of course possible that we made a mistake and rejected a research gem; it would not be the first time. A study of the peer review process and editorial decisions at three leading journals, including The BMJ, concluded that while “editors and reviewers generally made good decisions,” their evaluative strategies also appeared to increase the risk that they would reject “unconventional or outstanding work.”
Readers might like to know that all of this has ended amicably: the authors of the Meacock paper have apologised for releasing information about the previous review and the identity of one of the reviewers. When we contacted them, we learned they had already been in touch with Professor Freemantle, who had graciously accepted the apology they offered. Let’s hope things end as well with the junior doctors’ dispute.
Elizabeth Loder, Head of research, The BMJ.
José G. Merino, US clinical research editor, The BMJ.
Rubin Minhas, research editor, The BMJ.
Acknowledgements: We are grateful to the authors of the paper and the peer reviewers for agreeing to make the peer review process public in this case.