The massive consequences of the covid-19 pandemic are apparent in many regions with overburdened healthcare systems, including substantial increases in all-cause mortality, psychosocial, and economic consequences. [1] In the same way that regional healthcare capacity has been exceeded by the demands of the pandemic, so too has the scientific publishing system been overwhelmed. [2] We expect stakeholders and decision makers to rely on scientific evidence to guide decisions about the pandemic, but the urgent need for information means that the formal scientific evaluation process is being increasingly sidestepped. Prioritisation is being given to publication speed over quality. This, in turn, pushes important content discussions into the public space before proper scientific review and vetting of research has been conducted, which leads to confusion and a false sense of confidence in unverified findings. [3]

Scientific journals provide an important platform for guided scientific discussions and serve as repositories for scientific knowledge that should inform decision-making, improve clinical practice, and ultimately the health of the population. Scientific journals have already published tens of thousands of covid-19 related papers that undoubtedly reflect an enormous amount of extra work by peer-reviewers, editorial offices, and editors.

Early in the pandemic, it became clear that the traditional scientific evaluation and publishing structures reached their limits and could not reliably deliver fast and carefully peer-reviewed publications. As the number of submissions to reputable journals, especially fast-track submissions increased, many editors and peer-reviewers found themselves substantially less available due to involvement in patient care, family commitments, or their own covid-19-related grant writing or research involvement. These pressures may have resulted in rushed research conduct, reduced, and partly invalid scientific evidence, and wasted resources. [4-6] Furthermore, we have witnessed several problematic examples of scientific results on population health topics, which were communicated first via online and traditional media based on preliminary work posted in preprint repositories. Though we support transparency and the practice of preprinting, subsequent discussions in the lay media, on Twitter, or television programmes over articles with methodological and interpretation concerns that had not yet undergone peer review and editorial curation contributed to confusion among the public and in political discussions. [3,7,8] While we acknowledge the pressure on all sides to obtain and rapidly communicate relevant results, acting on premature, unvetted scientific evidence may result in potentially bigger harm than a delayed but valid publication. [3,4] 

These pressures on the scientific publishing system have not only affected covid-19 research. Similar to the neglect of non-covid-19 medical conditions, the processing of non-covid-19 related research submissions at many journals has been substantially slowed. [9] This has not only led to frustration of many scientists and consequences in the timing of delivery of scientific evidence, but also indirectly slows down the promotion processes of young scientists as these are often directly linked to the publication of scientific papers.

The pandemic exacerbates existing structural problems of scientific publishing. First, it widens the gap between resource-rich and resource-poor journals. [10] Research-rich journals are struggling to handle the major influx of articles and handle fast-track submissions in a timely manner. Resource-poor journals will grapple with the same problems with even fewer human resources to support editing, peer review, and publishing activities. Second, as peer review is generally uncompensated work conducted in spare time, delays due to increased research workloads and additional commitments outside of work, such as care of family and relatives, further reduces the pool of available peer reviewers. Since women carry much of the burden of these extra commitments, this may explain the comparatively low number of female authorships in covid-19 publications. [11] Because editors and peer reviewers are also researchers themselves, these structural problems are inevitable; during a global pandemic when researchers’ capacities are already overstretched, there is no way they can tackle the drastically increasing demand for peer review and editorial services.

While the peer review process certainly has its limitations and gaps, it will remain an important aspect of scientific discussion and exchange. [12] However, the covid-19 pandemic has intensified the importance of modernizing the scientific peer review and publication processes. New solutions are needed to ensure a timely bi-directional discussion between editors, peer reviewers, and authors, even in circumstances of low time resources, high supply of research papers, and high public pressure for speed. [12]

At the journal level, potential solutions might include:

• A two-track reviewing system of editorial triage that moves both pandemic-related and other submissions and forward
• Rapid initial assessment of research methodology before sending papers for full peer review, ideally by compensated methods-trained reviewers
• Sharing peer review comments with authors in real time, so that they can begin revisions even before all comments are in. Even if the article is ultimately rejected, this could foster improved quality of future submissions elsewhere. 
• Share peer review comments between journal upon submission to a second journal 
• Increased training in research methodology for peer reviewers and editors
• Innovative methods of peer review, such as those conducted by a small groups of scientists-in-training guided by a mentor
• Flexible decision-making process, such as daily short meetings rather than fixed weekly meetings to decide on submitted papers
• Hiring of temporary additional editorial staff 
• Support modification of inter-institutional academic promotion criteria to provide credit for peer review services, and other compensation for peer review services
• Prioritizing papers with openly available data and/or statistical analysis code would promote quality, reproducibility and transparency

The immediate and long-term consequences of covid-19 will occupy the scientific community for many years to come and the scientific publication platforms will certainly seek solutions to adapt to these challenges. With such a rapid influx of research publications, many of which are of questionable quality, the research community as well as the general public relies on the quality assurance and curating functions of traditional scientific journals. Though the existing peer review system has its flaws, modifying the current system seems superior to abolishing it entirely. We acknowledge that we are criticising a system we are integrally a part of. We have published fast track papers, declined some peer reviews, cite preprints, and we are now submitting yet another covid-related piece in which we publish about how busy journals are in a journal. We hope the utility of our suggestions outweighs the irony.

Tobias Kurth, Institute of Public Health, Charité – Universitätsmedizin Berlin.

Marco Piccininni, Institute of Public Health, Charité – Universitätsmedizin Berlin.

Elizabeth W. Loder, Brigham and Women’s Hospital, Boston, and Head of research, The BMJ. 

Jessica L. Rohmann, Institute of Public Health, Charité – Universitätsmedizin Berlin. 

 

Competing interests: All authors are passionate scientists and involved in the conduct and evaluation of scientific projects and peer-review training. TK, EWL and JLR conceptualized “Peerspectives”, a peer review training program for doctoral students in collaboration with The BMJ. TK is a consulting clinical epidemiology editor at The BMJ. EWL is head of research at The BMJ. Outside of the submitted work, TK reports to have contributed to an advisory board of CoLucid and a research project funded by Amgen, for which the Charité – Universitätsmedizin Berlin received unrestricted compensation. He further reports having received honorariums from Lilly, Newsenselab, and Total for providing methodological advice, from Novartis and Daiichi Sankyo for providing a lecture on neuroepidemiology and research methods. JLR’s position is funded by a grant from the Else-Kröner-Fresenius Foundation (www.ekfs.de, GSO/EKFS-17; TK was the grant recipient). She also reports outside of the submitted work to have receiving a research grant from SPOKES Wellcome Trust Funded Translational Partnership Fellowship for an evaluation study of the “Peerspectives” peer review training program. JLR further received financial support from Novartis Pharma for conducting a self-initiated research project on migraine remission in aging on a population level unrelated to this work.

 

References: 

1 Piccininni M, Rohmann JL, Foresti L, et al. Use of all cause mortality to quantify the consequences of covid-19 in Nembro, Lombardy: descriptive study. BMJ 2020;369:m1835.

2 Fagiuoli S, Lorini FL, Remuzzi G, et al. Adaptations and Lessons in the Province of Bergamo. N Engl J Med Published Online First: 5 May 2020. doi:10.1056/NEJMc2011599

3 Steinberg I. Coronavirus research done too fast is testing publishing safeguards, bad science is getting through. The Conversation 2020.http://theconversation.com/coronavirus-research-done-too-fast-is-testing-publishing-safeguards-bad-science-is-getting-through-134653 (accessed 10 May 2020).

4 Wynants L, Van Calster B, Bonten MMJ, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ 2020;369:m1328.

5 London AJ, Kimmelman J. Against pandemic research exceptionalism. Science 2020;368:476–7.

6 Glasziou PP, Sanders S, Hoffmann T. Waste in covid-19 research. BMJ 2020;369:m1847.

7 Yong E. Why the Coronavirus Is So Confusing. The Atlantic 2020. https://www.theatlantic.com/health/archive/2020/04/pandemic-confusing-uncertainty/610819/ (accessed 9 May 2020).

8 Motta Zanin G, Gentile E, Parisi A, et al. A Preliminary Evaluation of the Public Risk Perception Related to the COVID-19 Health Emergency in Italy. Int J Environ Res Public Health 2020;17. doi:10.3390/ijerph17093024

9 Rogers SK, Hughes M. Forgetting ‘routine’ DVT and stroke during COVID-19 is a parallel pandemic that will be costly if ignored. J Vasc Surg Published Online First: 30 April 2020. doi:10.1016/j.jvs.2020.04.486

10 Elizabeth Loder: Rich journals, poor journals, and why we should mind the gap. The BMJ. 2018.https://blogs.bmj.com/bmj/2018/04/09/elizabeth-loder-rich-journals-poor-journals-and-why-we-should-mind-the-gap/ (accessed 16 May 2020).

11 Andersen JP, Nielsen MW, Simone NL, et al. Meta-Research: Is Covid-19 Amplifying the Authorship Gender Gap in the Medical Literature? arXiv [cs.DL]. 2020.http://arxiv.org/abs/2005.06303

12 Tennant JP, Ross-Hellauer T. The limitations to our understanding of peer review. Res Integr Peer Rev 2020;5:6.