Guest Post: Sarah Wieten
The recent article, “Some Social Scientists Are Tired of Asking for Permission” in the New York Times inspired a great deal of debate about the role of institutional research ethics board (IRB) oversight in social science, which some argue is in most cases unlikely to involve significant harm to participants.
While the role IRBs play in sociological research is being re-examined, the importance of IRB oversight for medical research was not similarly called into question. But what exactly does IRB oversight in medical research involve? Should these groups be content with assuring that patients and participants in medical research have provided informed consent? Or do they have wider duties? What is the relationship between methodologically rigorous science and ethical science?
The approval of research projects by IRBs is an integral part of the conduct of research in universities. IRBs ensure that all research follows key ethical guidelines and is pursued for good reason, and in doing so, they aim to keep patients and participants out of harm’s way. IRBs are important gatekeepers of institutional research, and serve as a check on the work of scientists, physicians, and others who are pursuing new knowledge.
We would assume then, that people serving on IRBs have a clear understanding of relevant research design. That way, they can check the research for ethical issues stemming from the methodology. They can also make sure that methodologically poor studies do not proceed, as this would be an unethical waste of resources (and would put participants at risk without a reasonable prospect of gaining reliable knowledge in exchange).
In a recent paper, my colleagues and I asked medical IRB members to fill out a short quiz comprised of eleven true-or-false questions (appended at the end of this post) that tested basic knowledge of clinical research design. We contacted members of the Association of American Medical Colleges and the Public Responsibility in Medicine and Research group, via email, letters, and postcards over the course of seven months, in 2012-2013. In the end, 148 individuals completed the entire quiz. Surprisingly, only thirty-nine (26.5%) of them got all the answers right.
There are many possible explanations for this low rate of correct answers. One possibility, of course, is error on our part—there could have been problems with the quiz itself. Maybe the questions were not worded clearly enough or oversimplified a complex idea, generating confusion and therefore a wrong answer. Or, given the relatively low response rate of our study (20%), maybe our sample was skewed in some way.
On the other hand, if our findings are pointing to a true lack of knowledge among IRB members, then there is cause for worry. It would suggest that there are important gaps in the knowledge of IRBs that make it difficult to ensure that methodologically and ethically unsound studies do not proceed. There are ethical dimensions to methodological choices in science. As noted above, poorly done science is an unethical waste of resources and places participants in an unethical position of risk with little or no chance of the reward of new, helpful knowledge.
Based on these findings, we call for required, periodic systematic training on clinical research designs for IRB members. Training would help ensure that IRB members have sufficient knowledge to succeed in assessing research ethics submissions effectively, and that patients and participants are protected. While this proposal may seem burdensome or ill-advised — given that many IRBs are already overworked and undercompensated, resulting in additional delays to research — we believe it is worth pressing the issue. IRB members must be well trained both in matters of consent and the traditional ethical codes, but also in recognizing methodologically weak studies, which have their own ethical consequences.
Post Script: Here are the quiz questions we included.
1. A purpose of randomization is to create groups that have similar characteristics. (True)
2. Another name for a randomized controlled trial is a cross-sectional study. (False)
3. A purpose of randomization is to avoid selection bias. (True)
4. Another name for a randomised controlled trial is a cohort study. (False)
5. Randomized controlled trials provide more credible evidence of treatment than observational studies. (True)
6. A randomized controlled trial must have a placebo group. (False)
7. The purpose of a Phase 1 trial is to assess the benefit of an experimental treatment. (False)
8. The main purpose of Phase 1 study is to assess the safety of an experimental treatment. (True)
9. All Phase 2 studies must have a control group. (False)
10. Participants have to be randomized to treatment groups in an observational study. (False)
11. In an observational study, the investigator assigns patients to receive a particular treatment. (False)
Paper: Those Responsible for Approving Research Studies Have a Poor Knowledge of Research Study Design: A Knowledge Assessment of Institutional Review Board Members
Authors: Rahul Mhaskar, Elizabeth Barnett Pathak, Sarah Wieten, Thomas M. Guterbock, Ambuj Kumar and Benjamin Djulbegovic