Engaging patients, caregivers, and patient advocates in scientific pre-publication peer review is a relatively new approach for many journals and institutions. Patient inclusion in peer review is increasing as it appears to improve the usefulness and relevance of patient centered outcomes research. By reviewing manuscripts submitted for publication, patients can help shape the scientific landscape by advancing how study information and research findings are conveyed. Patient reviewers can also offer future research ideas for studying outcomes that are important to patients and enhance applicability of research findings. To further support their participation, tailored programmes that will facilitate patient participation in developing (i.e., writing, reviewing) scientific manuscripts and reports are needed.

Given that patient review programmes are still in their infancy, the utility of review comments provided by patient peer reviewers has yet been fully established. Still, a growing body of evidence suggests patient comments help authors improve quality of the research manuscripts. Since the inception of the Patient-Centered Outcomes Research Institute (PCORI) Peer Review Program in 2016, 175 patient, caregiver, or patient advocate reviewers have provided insights and recommendations on over 280 PCORI-funded research reports. From our experience managing peer review for PCORI, we have seen firsthand how patient reviews improve the quality of the research reports. For example, while commenting on a report about prescription pain management, a patient reviewer noted that the study was framed in the context of overall opioid dose reduction. This implied that alternative pain therapies could replace opioid treatment rather than presenting these alternative therapies as an adjunct to pain control with opioid treatment. Based on this feedback, the authors improved the report by adding a more balanced background describing the pros and cons of long term opioid treatment and a clearer description of curriculum intent, which made it more reflective of the broad range of patients’ experiences with pain management.

One of the factors affecting the usefulness of a reviewer’s comments is the quality of the review itself. Despite their promise, there is variability in the quality of patient reviewers’ comments. In our experience, the quality of a patient reviewer’s comments depends on the similarity of the patient’s personal experiences with the research study under review and the patient reviewer’s skillsets in interpreting and evaluating research texts. While most patients are not able to comment on the scientific or methodological rigor of the study design, there is great value in including the patient voice as their experiences further inform study dissemination. Indeed, a patient’s perspective can speak to the feasibility of adhering to a studied intervention in the real world, barriers that may affect access to an intervention, and the extent to which the outcomes in the study are important in helping to make a health related decision, all of which are invaluable contributions to the literature.

While there are numerous (see examples 1 and 2) instructional materials available for scientific reviewers (but not patient reviewers), little effort has been dedicated to developing resources and tools to support patient reviewers. To support patient peer reviewers for PCORI’s peer review, we developed a web-based training that provides videos, learning exercises, and additional resources to help patient reviewers write high-quality reviews. Our study provides details on our collaboration with patient reviewers that developed this training and suggests that a tailored training may improve patients’ knowledge of how to peer review research papers. Further, our study showed that after the training, patients could better recognize which review comments would be helpful to authors of the research reports and were more confident that they could complete high-quality peer reviews. 

Over 100 patient reviewers have accessed the PCORI online training since its inception in 2017. We have continued to gather feedback and updated the web-based training in October 2019 based on recommendation for a more interactive experience and more content on how to provide patient-relevant and useful comments. The training now includes additional videos from patient reviewers, updated exercises that distinguish comments on how the research was reported from comments on how the research was conducted, and offers users the opportunity to receive additional support from an experienced PCORI patient reviewer.

Including patients in the pre-publication peer review process is just beginning to become common practice for many journals and institutions. This is important as patients, caregivers, and patient advocates are eager to provide their unique feedback on manuscripts and reports that are typically not offered by traditional scientific reviewers. Equipping patients with the tools, resources, and training opportunities to support the writing of high-quality reviews is a worthy investment that could improve the usefulness and relevance of published research findings to a larger general audience.

Kelly J. Vander Ley, PhD is a Staff Scientist at Oregon Health & Science University (OHSU) and Research Associate at the Portland VA Research Foundation.

Competing interests: Dr. Vander Ley currently serves as the Deputy Director and an Associate Editor at the OHSU Editorial Office (Portland, OR, USA), which is under contract to manage peer-review services for the PCORI. The views expressed here belong to the co-authors and do not represent the views of PCORI, OHSU, AHRQ, or Kaiser Permanente.

Ilya Ivlev, MD, PhD, MBI is an Affiliate Investigator at Kaiser Permanente Northwest, Center for Health Research (Portland, Oregon, USA).

Twitter: @DRivlev

Competing interests: Between 2017 and 2019, Dr. Ivlev worked at the OHSU Editorial Office (Portland, OR, USA), which is under contract to manage peer-review services for the PCORI. Dr. Ivlev is currently supported by grant number K12HS026370 from AHRQ. The views expressed here belong to the co-authors and do not represent the views of PCORI, OHSU, AHRQ, or Kaiser Permanente.

Acknowledgment: We thank Marina Broitman (Associate Director for Peer Review at PCORI), Amy Price (Patient Editor for the British Medical Journal), Rebekah Webb (OHSU Editorial Office), and Kevin Lutz Richard X Martin (Kaiser Permanente Center for Health Research) for their suggestions on how to communicate this opinion piece.