Differential Attainment

Many medical educators ‘have been’ and ‘remain’ challenged by the concept of Differential Attainment (DA). The term DA describes a variation in levels of educational achievement that occurs between different demographic groups undertaking the same assessments. But this is not limited to medical education, Regan De Bere et al, have shown that DA is a common characteristic of ‘professional education’.

Widely reported definitions for DA include increased feelings of segregation owing to representing a ‘minority culture’ or ‘religion’; unsatisfied with the higher education experience compared to White colleagues; Typecast risk, for instance negatively stereotyped groups can experience increased anxiety at the prospect of compliance to a stereotype, leads to an underperformance in assessments.

There is some consensus that the explicit prejudice of examiner bias is unlikely to be the ‘sole’ cause, as anonymously marked ‘multiple choice’ exam findings have also demonstrated inconsistencies amongst White and Black, Asian, and Minority Ethnic (BAME) students. Yeates et al. also explored whether there was evidence of ‘examiner bias’s against BAME students in an experimental study,¹ the occurrence of bias was not substantiated.² However, employing a “lexical assessment assignment”, Yeates et al., confirmed the existence of ‘Asian typecasts’ in examiners’ way of thinking.¹  This was the exact opposite to the studies anticipations, this finding transpired irrespective of whether assessors detected ‘stereotype-consistent or -inconsistent in performances.’¹

However, clinical grades are understood to play a part in choice for ‘residency training’, furthermore Woolf et al., propose poorer clinical results for BAME students may contribute to their ‘underrepresentation’ in ‘residency training’ and more crucially in the healthcare labour force. A visible and consistent picture is also now emerging from research which adopted a cross disciplinary approach.³ The unease around DA has been substantiated by numerous studies that provide evidence, that ethnic minorities medical students have underperformed compared with those from the ethnic majority in clinical (rater-based) evaluations, despite controlling for prior attainment.⁴ BAME medical students are around ‘2.5 times’ likely to not pass an examination when matched to the White majority.^5,6

DA appears prevalent amongst ‘international medical school graduates’ (IMGs).⁷ This may be a linked to the difference between inequality and inequity, which has been seen to subsequently play a part in DA. This is especially noticeable when it comes to IMGs who actually need more support in their training for exams ‘not the same’ as their starting position is different. This can be due to cultural issues, understanding of language, nuances- as well as simple yet important things such as relevant textbooks, tips from colleagues, question banks etc DA is also seen throughout various assessment processes, plus ‘practical examinations’ (for instance Objective Structured Clinical Examinations – OSCE), ‘multiple-choice questions’ (MCQ) as well as pass-fail assessments.⁸ Comparable findings have been detected amongst medical students in the United States, the Netherlands and Australia.² Yet, doctors from BAME backgrounds make up a third of the National Health Service (NHS) workforce in the United Kingdom (UK). Indian doctors (11%) are a noticeable minority group, followed by doctors from Pakistan (5%).⁹

Preventative future interventions

To fully understand the effects and tackle DA, we need to stop tinkering at the sides and direct our struggles to change the core focus a on the issue that is, of the educational assessment itself and perhaps consider the very definition of what an examination sets out to achieve and for whom. We must honestly allow a ‘visualised’ picture with the proven factors that account for DA to emerge and understand that confronting DA is not only important from a ‘social justice’ standpoint, but also from a ‘service delivery’ perspective. In the UK, the NHS honours diversity and serves a diverse population. Moreover, the RCGP and BAPIO have identified, that failing to focus on inequalities can be costly and have legal and moral challenges.

Accordingly, underperformance can be attributed to a mismatch between the student and the educational environments that they become a part of, and justifications can be seen equally from both viewpoints, that is in terms of ‘them’ or ‘us?⁹ Stegers-Jager offers some crucial guidance on this, that when placed in an alternative way, “Do ethnic minority students perform worse or do their assessors think they do so?”.  This is highly relevant, to indicate “what” leads to racial and cultural disparities in performance and is a precondition for constructing interventions directed at safeguarding decent ‘clinical grades’ for BAME students. The reasons underlying DA in undergraduate medical students is uncertain and ‘examiner bias, prior educational attainment (e.g., A-Level grades), socioeconomic status and psychological factors have all not been able to fully explain the differences seen between White and non-White students”^2,9 Still, tackling the factors impacting DA requires appropriate considerations and posing proper questions of the institutions within which DA occurs. ‘How should assessment programs be constructed and regulated to encourage the right type of institutionalization, one that is based on fairness? This can occur by reviewing the objectives of examinations, the conditions of medical institutions, and the characteristics of colleges that favour or weaken calls to set the record straight on DA and explore the contextual causes.

The foundation and basic goal of assessment is to guarantee educational excellence and more importantly the outcome of ‘quality’. The approach taken by the “professional model” depends on several contributing factors; the significance of detailed specifications of the ‘providers of services’, and the assumption that the “professional’ s knowledge and code of ethics, supported by the vigilance of the profession, are the best guarantees of quality.” Conventional descriptions of professional excellence have been focused more on “input” and “process” measures rather than on “outcomes”  such as ‘quality’ however (…) the intricacy of professional assessment requires that the ‘professional community’ itself to be the decisive authority of ‘quality.’ Assessment now needs to promote a fair and equitable educational ‘quality’ by fostering a revamped professional code of conduct, one that acknowledges the importance of learning outcomes and accepts the principle and practice of accountability to BAME audiences, and the collective faculty must assume responsibility for the overall learning of BAME students. Institutional culture also has a powerful role as contextual causes of DA may indicate gaps in the institutional system. Given the culture and organizational environment of higher education, gaining acceptance for this revamped definition of professional responsibility is no small task and will require the will of other key performers and leaders in healthcare education.

References

1. Yeates P, Woolf K, Benbow E, Davies B, Boohan M, Eva K. A randomised trial of the influence of racial stereotype bias on examiners’ scores, feedback and recollections in undergraduate clinical exams. BMC Med. 2017 Oct 25;15(1):179.
2. Low D, Pollack SW, Liao ZC, Maestas R, Kirven LE, Eacker AM, et al. Racial/Ethnic Disparities in Clinical Grading in Medical School. Teach Learn Med. 2019;31(5):487–96.
3. Stegers-Jager KM. Is it them or is it us? Unravelling ethnic disparities in undergraduate clinical performance. BMC Med. 2017;15(1):190.
4. Stegers-Jager KM, Steyerberg EW, Cohen-Schotanus J, Themmen APN. Ethnic disparities in undergraduate pre-clinical and clinical performance. Med Educ. 2012;46(6):575–85.
5. Marshall IJ, Wolfe CDA, McKevitt C. Lay perspectives on hypertension and drug adherence: Systematic review of qualitative research. BMJ. 2012;
6. Goldacre MJ, Davidson JM, Lambert TW. Country of training and ethnic origin of UK doctors: Database and survey studies. Br Med J. 2004;329(7466):597–600.
7. Robinson DBT, Hopkins L, James OP, Brown C, Powell AG, Abdelrahman T, et al. Egalitarianism in surgical training: let equity prevail. Postgrad Med J. 2020;postgradmedj-2020-137563.
8. De Vet R, Van Luijtelaar MJA, Brilleslijper-Kater SN, Vanderplasschen W, Beijersbergen MD, Wolf JRLM. Effectiveness of case management for homeless persons: A systematic review. American Journal of Public Health. 2013.
9. Woolf K, Mcmanus IC, Potts HWW, Dacre J. The mediators of minority ethnic underperformance in final medical school examinations. Br J Educ Psychol. 2013;83(1):135–59.

Authors

Dr. Nagina Khan, BHSc, PGCert, Ph.D.

Nagina is a Senior Clinical Research Fellow in Primary Care, Centre for Health Services Studies (CHSS), Division of Law, Society and Social Justice, School of Social Policy, Sociology & Social Research, University of Kent. She is the CHSS PGR Lead (interim) and Director of the MSc Applied Health Research Programme, University of Kent.

Nagina’s current research supports the Integrated Care Systems (ICS) to capitalise on emerging existing networks in its research duty and mitigate the current risk of future research being conducted in silos and without focus on priorities and underserved populations. This work will diversify the public voice listened to and strengthen ICS strategic links within local research infrastructure to support evidence-based practice, apply solutions, and spread innovation.

Nagina was a senior postdoctoral researcher and Visiting Researcher, Department of Psychiatry, University of Oxford. She has worked as a Scientist at Centre for Addiction and Mental Health (CAMH). Nagina also worked with Touro University Nevada, Las Vegas, US, and with the Royal College of Psychiatrists. Nagina was a Medical Research Council (MRC) Research Training Fellow, her research was centred on complex interventions for people with depression, University of Manchester. Her post-doctoral studies were undertaken at the NIHR School for Primary Care Research, UK focusing on First episode Psychosis in young people using Early Intervention Services. Nagina’s research interests include Medical Education, Professionalism, Social Justice in Healthcare, Complex Interventions for Depression, First episode Psychosis in Young People, Culturally Appropriate Mental Health Care, Women’s Mental Health, Incentivisation Schemes (P4P) in Healthcare for HICs and LMICs and Global Health. Nagina is the Associate Editor at BMJ Mental Health, she is also the BMJ Leader Editorial Fellow and was an Editorial Board Member of the BioMed Central Medical Education Journal.

Professor Partha Kar

Partha Kar is the Type 1 Diabetes & Technology lead for NHS England and co-lead of the Getting it Right First Time (Diabetes) and Consultant, Diabetes & Endocrinology in Portsmouth.

In his role with NHS England, he has helped to expand use of technology in Type 1 Diabetes – namely use of Flash Glucose/CGM & implementation of use of CGM in T1D pregnancy along with use of online digital self-management platforms, while recently leading on real world data collection on Closed Loops for subsequent NICE review. He has worked subsequently with NICE on updating relevant guidelines in non-invasive glucose monitoring access in Type 1 and Type 2 Diabetes.

Partha has been involved in the introduction of frailty into Quality Of Framework (QoF) treatment targets, Diabulimia pilot projects in the NHS; championing “Language Matters” and helping to create an overview of Diabetes care in Primary Care Networks. Recent work has included principles of Peer support for Type 1 Diabetes as well as a Decision Support Tool for those with Type 1 Diabetes.

He has also been recognised as one of the most influential figures from the ethnic minority population across healthcare in the UK by the Healthcare Service Journal in 2020,2021, 2022 & 2023 and received an OBE for services to Diabetes care in 2021.

Professor Subodh Dave

Subodh is an international medical graduate having done his MD and DNB (Psychiatry) from Grant Medical College, Mumbai, India. He moved to the UK in 1995 and obtained his CCT in General Adult Psychiatry with an endorsement in Liaison Psychiatry. He works as Consultant Liaison Psychiatrist in Derbyshire Healthcare Foundation Trust and is Professor of Psychiatry at the University of Bolton. He is Deputy Director of Undergraduate Medical Education and in that role has led innovations in introducing and embedding simulation and lived-experience involvement in the training of medical students at the University of Nottingham.

Subodh has held training roles at all levels spanning undergraduate, foundation and postgraduate training both in the UK and internationally.

He is passionate about ensuring that training, assessment structures and CPD (Continuing Professional Development) programmes lead to improvements in patient care and clinical outcomes.

Declaration of interests

We have read and understood the BMJ Group policy on declaration of interests and declare the following interests: none.