Transparency has been at the heart of BMJ Open for its entire ten year history and a key component has been to operate a fully open peer-review system. We believe that this approach is the most equitable way of making the peer-review process a fair and collaborative endeavour.
Concerns about single-blind review, which has traditionally been
the standard method of peer review, where the identity of the reviewer is hidden from the author and readers are well documented. Both anecdotal and experimental evidence have shown that biases against sex, nationality, institutions, or caustic unconstructive comments could be easily hidden under the cloak of anonymity.1 It would be easier to sabotage potential competitors if they were unaware who was reviewing their paper. From a reviewer’s perspective, their hard work and time spent helping to improve a manuscript can go unrecognised if their identity is concealed.
Peer review systems
The two most obvious systems to replace single-blind review are double-blind review, where the identities of both reviewers and authors are concealed, and fully-open review where both reviewers’ and authors’ identities are known to one another. The former has practical difficulties such that it may be relatively easy to deduce who has authored the paper, which restores the process to a single-blind review. Furthermore, the reviewer still gets no recognition. Trials showed no benefit to the double-blind system compared with single-blind review.2,3 Triple-blind review systems have been proposed whereby the editors are also blind to the authors’ and reviewers’ identities, but this would carry the same limitation of editors or reviewers being able to identify authors (or at least attempting to do so), and create difficulties for journals and editors as to how to assign reviewers, spot conflicts of interests and ensure appropriate expertise of reviewers.
The BMJ decided in 1999 to start adopting an open-peer review approach, albeit implemented in stages, beginning with sharing reviewer names and working up to publishing reviews and revision history alongside the articles.4 BMJ Open launched with fully open peer review in 2011 and The BMJ began publishing reviews and article histories in 2014.5
Such a system where the reviews and reviewers’ names are published alongside their comments might be expected to produce a higher quality review, but a study by The BMJ found no overall difference whether the reviewers expected to have their names and comments published or not.6 Nevertheless, this research provided no reason to change from the system in place and the ethical benefits were considered sufficient to outweigh disadvantages, such as some reviewers being reluctant to accept. Hopefully, as open review becomes more prevalent concerns will diminish.
An editor’s perspective
Perhaps we can accredit the lack of substantial difference in review quality across the three systems to the integrity of reviewers. As editors we have all seen those reviews that seem antagonistic and unconstructive, but thankfully they are very few and far between—although they do stick in the mind. Far more often, we do see good quality reviews. But what constitutes a good review?
From an editor’s perspective, the most helpful review would include comments on the strengths of the paper as well as the areas that could be improved. Advice for improvement should be clear and constructive, with suggestions or specific comments about what is wrong and could be done to correct or improve the study. From an author’s point of view this also gives a clear view of what the reviewers think should be done to improve the paper or good understanding of why it is being rejected, if that is the case. At BMJ Open we believe that the review process gives the opportunity to apply and share their expertise in a professional activity and that it is right that reviewers should be recognised for their efforts. In turn, their institutions are recognised for the sphere of influence generated by their experts.
An open and beneficial system
Although broad support for aspects of open peer review have been shown, the situation is complex.7 More researchers were supportive of publishing review reports than for sharing the identity of the reviewers. Nevertheless, we think that a more open and beneficial relationship can be established between reviewers and authors that outweighs the probable very small risks of a negative effect. Suggestions that in an open review system, prior review history could influence later decisions were observed to be unfounded,8 and although the same study found a possible tendency to be more favourable to authors from the same country the evidence was extremely weak. In any case, such a marginal bias could potentially occur in single-blind review journals.
We believe that open peer review has been a key element of the success of BMJ Open over the past 10 years and we are happy to embrace this system as we continue to publish quality clinical research.
References
1 Groves, T. Is open peer review the fairest system? Yes. BMJ 2010;341:c6424
https://www.bmj.com/content/341/bmj.c6424
2 van Rooyen S, Godlee F, Evans S, Smith R, Black N. Effect of blinding and unmasking on the quality of peer review: a randomised trial. JAMA 1998;280:234–237.
https://jamanetwork.com/journals/jama/fullarticle/187750
3 Justice AC, Cho MK, Winker MA, Berlin JA, Rennie D, the PEER investigators. Does masking author identity improve peer review quality: a randomised controlled trial. JAMA 1998;280:240–242.
https://jamanetwork.com/journals/jama/fullarticle/187758
4 Smith R. Opening up peer review. BMJ 1999;318:4
https://www.bmj.com/content/318/7175/4.short
5 Groves, T and Loder E. Prepublication histories and open peer review at The BMJ. BMJ 2014;349:g5394.
https://www.bmj.com/content/349/bmj.g5394
6 Van Rooyen S, Godlee F, Evans S, Black N, Smith R. Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. BMJ 1999;318:23-7.
7.Ross-Hellauer T, Deppe A, Schmidt B. Survey on open peer review: Attitudes and experience amongst editors, authors and reviewers. PLoS ONE 2017;12:e0189311
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0189311
8 Thelwall T, Allen L, Papas E-R, Nyakoojo Z, Weigert, V. Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000Research post-publication open peer review publishing model J Inf Sci 2020. DOI/10.1177/0165551520938678
https://journals.sagepub.com/doi/10.1177/0165551520938678