“Are numbers in quantitative research objective?”
Our 2021-2022 Masters of Global Health class at Karolinska Institutet reflected during an in-class discussion on the role of reflexivity in quantitative research. The question was raised after learning about the concept in our qualitative research lectures. While reflexivity was used as a quality measurement in qualitative research, we noted that the concept was not being taught for quantitative research. We wondered why this was the case and concluded that this should change. Within our diverse class, with backgrounds ranging from the clinical fields to social sciences, differential ideas merged towards a common thread, which we present in this commentary.
Reflexivity is the practice of the researcher examining their research practices and critically reflecting on their own role. This process occurs in two dimensions, first the researcher should reflect about why the research question is being asked, what the interest is and why it attracts funding. Secondly, who is the researcher themself, how do their characteristics (e.g. gender, country of origin) shape the research methods (Green and Thorogood 2018). As future Global Health professionals educated in a high-income country focusing on health topics in diverse settings we see reflexivity as an opportunity to reflect on and define our roles and the influence of our background on our work. Changing from a qualitative to a quantitative perspective during our course gave us an insight into the different principles of the two methodologies. While appreciating the specific role of each methodology in evidence generation, we asked ourselves: are numbers objective? Are they not impacted by the role of the researcher themself? And therefore, do we not need reflexivity when working with this data format?
“While carrying out quantitative research you notice that there is some sort of grey zone, you have to make choices”
What emerged in our discussion is that while some might argue numbers are objective in themselves, the research process of obtaining those numbers and results is not. Many choices are made along the way, and the researcher’s characteristics, role, personal experiences and position define the lens through which they see the project and can therefore influence the outcome and interpretation of the findings. Even when researchers are using the same research question and data set, large variations in their conclusions can be seen due to their different choices in regards to data cleaning and types of analyses used (Huntington-Klein 2021). Statements of limitations and the application of methodological checklists aim to increase transparency. However, we believe that reflective practices addressing the two dimensions described earlier, could add to the quality of quantitative research. Accounting for some of these differences in conclusions between attempted replications, rather than presenting the results as a single objective truth could also address inequities and ethnocentrism in research.
We are missing a formalized process within quantitative research and current syllabi that allows to reflect on the researcher’s role and influence on a project. In our discussion we elaborated on the guidelines developed by Morton et al. on reflective statements in publications by incorporating our own desires to learn about reflexivity in the context of quantitative research. Reflexivity has been avoided in quantitative research as it was considered to decrease credibility, this perspective was fuelled by the dichotomy between quantitative and qualitative research. We argued that including Reflexivity in quantitative studies is rather a way of strengthening the credibility of this research. Complex interactions and communication between researchers and participants also exist in quantitative studies and reflexivity can add value by exploring their impact on the research process (Ryan and Golden 2006). This is especially true when knowledge is seen as a fluid concept that is subject to change. We identified three key points where we see a need for reflective practice and want to thrive for improvement (Box 1). The first reflexivity moment should occur before the research is carried out, the researcher should examine their motivation for the project and if it is ethical and norm critical. While conducting the project it is important to consider why specific methodology was chosen, why minimising some biases was prioritized over others and subsequent influences on the data. Lastly after the research has been finalized the researcher should once again examine their role, how their project contributes to scientific perspectives and what the benefits are.
We agree with current calls for reflective statements and the provision of a structure for reflexivity in publication practice. But crucially, methodological reflexivity needs to be incorporated into future researcher and global public health professional training for quantitative research. As students, we feel this perspective would support us in promoting quality and equity for our field, and biases and ethnocentrism could be reduced.
| Key points | Considerations |
|---|---|
| Before | On which pre-assumptions is the project based? How does my personal background influence the research question? Motivations Why am I personally interested? What is the benefit and who will benefit? Who is funding the project and why? Ethics Were inequalities considered? Theory Was a range of diverse theory applied? How were research priorities set? Conflict of Interest Who are the stakeholders and how are they interconnected? |
| During | Methods Why were certain methods chosen over others? Are there changes made during the process? Data Collection How could the data be influenced by the data collectors or the context of data collection? |
| After | Introspection Who was involved in data analysis and interpretation? How did this influence the analysis? What other sources of information support the interpretation? Reflection What are the learned lessons? What do researchers personally take away from this project? |
About the authors: Theresa Pfurtscheller is a Medical Doctor from Germany with research experience in Medical Education through her doctorate and clinical experience in children’s psychiatry and paediatrics. Currently she is a Global Health Master student at Karolinska Institutet and her special interest lies in the field of Global Child Health.
Anja Wiemers is a Medical Doctor, educated in the Netherlands, and is currently a Global Health Master student at Karolinska Institutet. She is passionate about universal health coverage and has been active in the access to medicine movement for multiple years.
Co-authors and additional contributors:
MSc GH KI 2021/2022
Katrine Thorgaard-Rasmussen [Midwifery, Denmark]
Maike Katharina Hentges [European Public Health, Germany]
Fanny Brattström [Medical Doctor, Sweden]
Hannah Casey [Consulting, UK]
Cole Ferguson [Microbiology and Cell Science, USA]
Isabel Smith [Experimental Psychology, UK]
Karim Abou Nader [Neuropsychology, Lebanon]
Aneesa Mahommed Saleem Solwa [Medical Doctor, South Africa]
Katerina Crawford [Molecular and Cell biology, USA/South Africa]
Course leaders Research Methodology (MSc GH, KI, 2021/2022)
Carina King, epidemiologist
Helle Mösted Alvesson, medical anthropologist
Handling Editor: Neha Faruqui