Mona Nasser: How can research publication be improved?

The first day of the EQUATOR/REWARD conference coincided with the publication of the most recent article on reducing waste in research—“increasing value and reducing waste in biomedical research: who’s listening?” The paper explores how different stakeholders responded to the call for “reducing research waste.” This initiative started with a short paper by Iain Chalmers and Paul Glasziou on avoidable waste in production of research evidence in 2009, followed by a more comprehensive series of articles in 2014. The National Institute for Health Research (NIHR) in England is one of the key organisations which responded to the change shortly afterwards. The quick response is promising. Doug Altman, the first plenary speaker, highlighted that throughout history change happened frustratingly slowly. He quotes Drummond Rennie, who said in JAMA in 2001:

If physicians are to base treatment decisions on the evidence in the medical literature, all the relevant results of trials must be available easily and consistently. Yet it is common to have trouble identifying the hypothesis, the research question, and the design of a published trial. It is even more common to lose count of the participants or to be unable to tell who received what therapies and the type of analysis used. As a result, it is often impossible to know whether the conclusions are justified by the data.

The current status of medical research literature is depressing. Research can only be useful if the methods are valid and the report is published in a useful format. However, there are examples going back to 50 years ago showing that this has not been the case. In 1966 Schor and Karten reported that “Thus, in almost 73% of the reports read (those needing revision and those which should have been rejected), conclusions were drawn when the justification for these conclusions was invalid.” Altman suggests that our mantra for research publication should be accuracy, completeness, and transparency. He describes the five taxonomies of bad reporting—unpublished research, selective reporting, incomplete reporting, misleading reporting, and finally inconsistency between sources. For example a survey in the reporting of epidemiological studies by Pocock showed an uneven distribution of p values in article’s abstract. Ten articles reported P value over 0.05, 23 articles reported P values below 0.0001 and 26 articles reported P values between 0.01 and 0.05. The authors concluded that there is a tendency to over-interpret arbitrary cut offs such as P < 0.05. Altman finished his talk with one way that they have tried to tackle the problem—reporting guidelines.

Following Doug Altman, Liz Wager reminded us that the responsibility to improve research publication lies with all of us. Researchers and journal editors are responsible, however, other stakeholders such as research institutes, regulators, funders, communication experts, and users need to be involved to drive change. Publishing research does not only incorporate journal papers. We publish part of our information/data in clinical data registries, submit proposals to an ethics committee, and submit data to regulatory agencies, etc. All these data need to be better linked and made openly accessible. The idea of better linking of information is not new. Iain Chalmers and Doug Altman wrote about a similar concept in 1999 called threaded publications:

Electronic publication of a protocol could be simply the first element in a sequence of ‘threaded’ electronic publications, which continues with reports of the resulting research (published in sufficient detail to meet some of the criticisms of less detailed reports published in print journals), followed by deposition of the complete data set.

During the question and answer session, a question was raised about whether we have a chance to change the policies of stakeholders like for example regulators. Wager argued that public engagement and campaigning can play a key role in demanding transparency from regulators.

Afterwards, David Moher talked about the responsibility of editors in this process. Reporting guidelines have been effective to drive changes, but we need more research on their impact. To achieve this and other similar researcher questions, we require a comprehensive mechanism to fund journalology.

Editors should work with publishers to push them to set up such funding mechanisms. Moher also reminded us that a lot of editors never get training on how to run a journal. We all need a driving license to drive; do we need a certification process for editors of journals? Moher and Altman introduced the concept of core competencies for editors in a recent article in PLOS medicine. Other key areas that editors can contribute to are:

  • implementing data sharing;
  • developing a core template for instruction of authors as the variety in current instructions is confusing for authors;
  • working together with funders in implementing SPIRIT guidelines for reporting protocol of clinical trials.

During the question and answer session, Trish Groves highlighted that the ICMJE offers standard guidelines for instructions to author but it isn’t uniformly implemented. The implementation of effective interventions was the topic of the final talk of the session by Ana Marusic. She reminded us that good editorial processes can cost lots of time and money and this is a big challenge for small journals. She suggested that we need simple tools to provide training and improve reporting standards like the online writing air tool for trial report (COWEB). She finished with a quote from Hugh Clegg:

“A subject that needs reform should be kept before the public until it demands reform.”

• View a storify of tweets from the first day of the conference here.

Mona Nasser is the clinical lecturer in Evidence Based Dentistry in Plymouth University Peninsula Schools of Medicine and Dentistry in Plymouth, England. She is also the co-convenor of the Cochrane Agenda and Priority Setting Methods Group.

Competing interest: MN is a member of steering group committee of the Cochrane Collaboration and Evidence Based Research network. She is also a coauthor of the most recent Lancet article on research waste ““Increasing value, reducing waste in biomedical research: who’s listening” that provides an initial evaluation of the Lancet’s 2014 series.