Journals are key stakeholders in the quest for greater transparency and reproducibility of research. Georgia Richards, Aidan Cashin, Matthew Bagg, Elaine Toomey, James McAuley and Hopin Lee discuss their recent publication that evaluates the policies and standards of pain journals.
Blog entry written on: Limited engagement with transparent and open science standards in the policies of pain journals: a cross-sectional evaluation (bmjebm-2019-111296)
Georgia Richards, Aidan Cashin, Matthew Bagg, Elaine Toomey, James McAuley & Hopin Lee.
Scientific progress is facilitated by research that is conducted and reported transparently and openly. The ability to replicate, validate and critically appraise research depends on a detailed account of the methods and analysis, and complete reporting of results. Journals play a key role in supporting authors to practice openly, which generally makes their research more transparent.
Encouragingly, pain journals have initiated conversations about open science. During the publication of a 2018 review on the reproducibility and replicability of pain research, PAIN published an editorial titled “Publishing the best basic and applied pain science: open science and PAIN”. However, we have not seen actual policy change at PAIN. Our latest evaluation published in BMJ Evidence Based Medicine, Limited engagement with transparent and open science standards in the policies of pain journals: a cross-sectional evaluation (bmjebm-2019-111296), shines a light on areas that can be updated and improved by pain journals.
Why pain journals?
Pain conditions are among the top three most common reasons for seeing a doctor. Since 1998, there has been a 153% increase in “pain” articles indexed in the medical subject headline (MeSH) on PubMed (see Figure 1). Encouraging and improving the transparency and openness of pain research has the potential to significantly improve how we perceive and manage pain.
Figure 1: Number of articles indexed in the “pain” medical subject headline (MeSH) on PubMed from 1940 to 2018, data were extracted on 28/01/2020 and the relevant pain articles are available here.
What did we do?
We evaluated and graded the policies of the 10 highest-ranking pain journals. We used two tools developed from the Transparency and Openness Promotion (TOP) guideline, and the International Committee of Medical Journal Editors (ICMJE) disclosure of interest requirements. The Centre for Open Science (COS) developed TOP in 2015 to encourage and improve the policies of journals (more details are here).
What did we find?
Overall, we found a low level of support for transparent and open practice in the policies of pain journals. Of the 10 journals, only one (Journal of Headache and Pain) encouraged the preregistration of non-clinical trials. Three (Journal of Pain, Molecular Pain and Journal of Pain and Symptom Management) were signatories of the TOP guideline, although, no journals allowed the submission of Registered Reports. Only half encouraged the sharing of data and code. Most (80%) journals required preregistration of clinical trials and 70% encouraged the use of reporting guidelines.
The Journal of Pain and the Journal of Headache and Pain had the highest TOP scores (7/29), see Table 1. The Clinical Journal of Pain (0/29) and Pain Medicine (1/29) had the lowest TOP scores. The Clinical Journal of Pain (3/4), the European Journal of Pain (3/4) and the Pain Physician (2/4) did not meet all four ICMJE requirements.
Table 1: Results from Cashin et al. 2020 [3] published in the BMJ Evidence Based Medicine. We ranked the top 10 pain journals by their policy commitment to the Transparency and Openness Promotion (TOP) guidelines and The International Committee of Medical Journal Editors (ICMJE) disclosure of interests statement.
|
Journal |
TOP score (/29) |
ICMJE disclosure of interest requirements ( /4) |
| Journal of Pain | 7 | 4 |
| Journal of Headache and Pain | 7 | 4 |
| European Journal of Pain | 5 | 3 |
| Journal of Pain and Symptom Management | 5 | 4 |
| PAIN | 4 | 4 |
| Molecular Pain | 3 | 4 |
| Pain Physician | 3 | 2 |
| Regional Anaesthesia and Pain Medicine | 2 | 4 |
| Pain Medicine | 1 | 4 |
| Clinical Journal of Pain | 0 | 3 |
Conclusions
The policies of pain journals do not encourage transparent reporting and the sharing of methods, materials and data. This is concerning because opaque scientific practices can result in unnecessary costs, slow scientific progress and may put patients’ lives in danger. The scientific and clinical communities should work together to raise awareness and shift the culture to one of transparency and openness. Researchers can integrate new tools and platforms into their workflows (e.g. the Open Science Framework (OSF) and GitHub). Authors and readers can actively engage with journals to encourage updating policies that will facilitate better science (e.g. the RegisteredReportsNow! initiative). If the policies of pain journals are updated to encourage transparent and open research, scientific progress and, ultimately, people with pain will benefit.
Who are we?
We are a group of researchers, academics and clinicians working collaboratively as OPeRA: Open Pain Research Advocacy and Appraisal. We are based in Europe and Australia and work in different areas of pain research and practice. Follow our work here.
Authors
Georgia C. Richards, University of Oxford, United Kingdom.
Short bio: Georgia is a Doctor of Philosophy (DPhil/PhD) candidate in the Centre for Evidence-Based Medicine at the University of Oxford.
COI: Georgia is funded by the National Institute for Health Research (NIHR) School for Primary Care Research (SPCR), the Naji Foundation and the Rotary Foundation. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. Georgia is an Ambassador for the Centre for Open Science and is the Editorial Registrar for BMJ Evidence Based Medicine.
Twitter: @Richards_G_C
Aidan G. Cashin, University of New South Wales, Australia.
Short bio: Aidan is a Doctor of Philosophy (PhD) candidate in the Centre for IMPACT at Neuroscience Research Australia, The University of New South Wales.
COI: Aidan is funded by the University of New South Wales Prince of Wales Clinical School Postgraduate Research Scholarship and a NeuRA PhD Candidature Supplementary Scholarship. Aidan is a Catalyst for the Berkeley Initiative for Transparency in the Social Sciences and received travel support from Memorial University of Newfoundland to speak about engagement with research.
Twitter: @aidancashin
Matthew K Bagg, University of New South Wales, Australia.
Short bio: Matthew is a Doctor of Philosophy (PhD) candidate in the Centre for IMPACT at Neuroscience Research Australia, The University of New South Wales.
COI: Matthew is funded by a NeuRA PhD Candidature Scholarship and Supplementary Scholarship. Matthew is an Ambassador for the Centre for Open Science and received travel support from Memorial University of Newfoundland to speak about engagement with research.
Twitter: @matthewbagg
Elaine Toomey, National University of Ireland Galway, Ireland.
Short bio: Elaine is a Research Fellow and Associate Director of Cochrane Ireland, within Evidence Synthesis Ireland based at the National University of Ireland Galway.
COI: Evidence Synthesis Ireland is funded by the Health Research Board of Ireland and the Public Health Agency of Northern Ireland. Elaine is a Catalyst for the Berkeley Initiative for Transparency in the Social Sciences.
Twitter: @elainetoomey1
James McAuley, NeuRA and University of New South Wales, Sydney, Australia
Short bio: James is a clinical researcher, director of the Centre for Pain IMPACT at NeuRA and Associate Professor in the School of Medical Sciences at UNSW.
COI: James’ position is funded by UNSW. He receives research funding from the Australian National Health and Medical Research Council, the Medical Research Future Fund, Mindgardens Research Fund and Sydney Partnership for Health, Education, Research and Enterprise (SPHERE) Maridulu Budyari Gumal.
Twitter: @pain_neura
Hopin Lee, University of Oxford, United Kingdom.
Short bio: Hopin is a postdoctoral research fellow at the Centre for Statistics in Medicine at the University of Oxford.
COI: Hopin has received funding from the Australian National Health and Medical Research Council and the Berkeley Initiative for Transparency in the Social Sciences.
Twitter: @hopinlee
DISCLAIMER
The views and opinions expressed on this site are solely those of the original authors. They do not necessarily represent the views of the BMJ and should not be used to replace medical advice. All information on this blog is for general information, is not peer-reviewed, requires checking with original sources and should not be used to make any decisions about healthcare. No responsibility for its accuracy and correctness is assumed by us, and we disclaim all liability and responsibility arising from any reliance placed on such commentary or content by any user or visitor to the Website, or by anyone who may be informed of any of its content. Any reliance you place on the material posted on this site is therefore strictly at your own risk.