Paul Glasziou and Iain Chalmers: Funders and regulators are more important than journals in fixing the waste in research

Funders and regulators have the principal power to implement most of the solutions needed to reduce research waste


paul_glasziou2Iain-ChalmersThe current level of waste in health and medical research has been estimated to be over 85% of the nearly $200 billion annual global spend [1]. That amounts to around $500 million per day—equivalent to the annual budgets of several prestigious research institutes. That level of waste should be an international scandal. But it is hard to point a finger of blame at any single person, group, or entity.

So who is responsible, and who may be able to “fix” the waste? Some commentators consider that journals are central to fixing the problems of research waste. [2] We agree journals can help publicize the size, causes, and possible solutions, and are a central part of good reporting. But journals come too late in the research production process to have much influence on four of the five stages of waste: the choice of research questions, the quality of design, the efficiency of conduct, and the submission of final reports, and have most influence on the fifth stage of complete and clear reporting. We will of course cheer on the efforts of journals, institutions, researchers, and others to reduce waste in research, but we suggest that it is funders and regulators who have the principal power to implement most of the solutions needed.

Consider non-publication of studies, which consistently averages around 50% across countries, sectors, institutions, and study designs. More than thirty years ago John Simes proposed universal registration of clinical trials as an essential part of the solution [3]. Since then trials registries have proliferated. This is good news given that registries are essential elements of the infrastructure needed to achieve Simes’ proposal. And there does appear to have been some slow improvement. This has involved many agents: journals that ask for registration, campaigns such as AllTrials, meta-researchers who study non-publication, and others.

The most notable examples of progress, however, have been: (i) the 10% of additional study results now available on ClinicalTrials.gov since introduction of regulations that required US trials to post results; [4] and (ii) the 98% publication rate achieved for trials funded by the UK NIHR’s Health Technology Assessment programme, using several means, including withholding the final 10% payment of the study grant until the full report has been made available. These activities have been taken by research funders, who not only provide essential infrastructure for the registries, but are also best placed to monitor and provide help, incentives, and, if necessary, regulations and penalties, to achieve full reporting of results.

Of course there is much to do beyond clinical trials registration and reporting. For example, animal research has been less researched, but similar problems of poor design, non-publication, and poor reporting seem to exist in that field of research. A recent study of 100 animal research studies presented at the 2008 Society of Critical Care Medicine Conference found that only 62% had subsequently been published. [5] Registries of animal studies would help monitor this form of scientific misconduct, but would also provide a searchable location for unpublished studies. However, on the basis of achievements within clinical research, funders would need to provide infrastructure and require study registration.

The story of study registration illustrates that individuals from many sectors need to play a part. But it also illustrates that research funders and regulators are pivotal to providing the means (registries) and motives (incentives and regulation) to achieve this approach to reducing waste. This pivotal role of funders and regulators is likely to be true of the other stages of research waste as well: the research questions addressed; good study design; efficient conduct; and timely and full reporting.

Given funders’ pivotal role, we have been delighted to see the formation of an international research funders’ forum to address what can be done to reduce waste and improve the value of the global investment in health and medical research.

Research regulators—the other source of power in addressing research waste—have begun meeting to explore how they, too, might establish a forum comparable to that created by research funders. Indeed, funders and regulators could play complementary roles. For example, the National Institute for Health Research—who were awarded the first REWARD prize—has addressed the other principal source of waste—embarking on new research without assessing systematically what is already known. Applicants to NIHR for funds to do additional primary research are now required to show by reference to systematic reviews of existing evidence and information about ongoing trials that proposed new studies can be justified.

Research funders and research regulators can continue to make progress in agreeing principles and practices leading to increased value and reduced waste in research. Journals and others inside and outside the research community can play their roles by helping to foster an atmosphere in which these changes are seen as necessary, not only for scientific, ethical, and economic reasons, but also to improve public trust in the research community.

Paul Glasziou is professor of evidence based medicine at Bond University and a part time general practitioner.

Competing interests: None declared.

Between 1978 and 2003, Iain Chalmers helped to establish the National Perinatal Epidemiology Unit and the Cochrane Collaboration. Since 2003 he has coordinated the James Lind Initiative’s contribution to the development of the James Lind Alliance, the James Lind Library, Testing Treatments interactive, and REWARD.

Competing interests: IC declares no competing interests other than his NIHR salary, which requires him to promote better research for better healthcare.

References:

  1. Research: increasing value, reducing waste, series at: www.thelancet.com/series/research
  2. Munafò MR, Nosek BA, Bishop DMV, et al. A manifesto for reproducible science. Nature Human Behaviour 1, Article number: 0021 (2017).
  3. Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol. 1986 Oct;4(10):1529-41.
  4. Chen R, Desai NR, Ross JS, Zhang W, et al. Publication and reporting of clinical trial results: cross sectional analysis across academic medical centers. BMJ. 2016 Feb 17;352:i637.
  5. Conradi U, Joffe AR. Publication bias in animal research presented at the 2008 Society of Critical Care Medicine Conference. BMC Research Notes201710:262 https://doi.org/10.1186/s13104-017-2574-0
  • Woody Caan

    It is a few years since I was involved in research funding decisions, but one weak link in the research chain is the review of proposals. An ideal reviewer would combine a broad familiarity with the health topic to be researched, wide personal experience of different methods and a capacity to prioritise proposals within the remit of each funding body that they advise. Such ideal reviewers probably do not exist (and if any do exist, they must be kept very busy by the demand for “expert” comments). The Medawar Curve of researcher effort (=investment) against originality (=scientific value) predicts that many researchers propose work that is too easy (trivial) or too hard (never completed). A good reviewer needs discernment that a proposal is worth doing and likely to lead to really new knowledge.

  • Сергей Яргин

    All professionals should participate in fixing the waste in research – everybody who has reasons to suspect unreliable data, biases, conflicts of interest, scientific misconduct should react by writing letters to the editors, comments etc.: https://www.researchgate.net/publication/312376851_Scientific_misconduct_and_related_topics