Interviews: Asking the wrong questions by Roger Kline

The NHS is awash with Action Plans to create more diverse recruitment and career progression and melt the snowy white peaks of the NHS which still symbolise the failure of the NHS to tackle race discrimination. Post Gorge Floyd, such efforts increased but I suggest progress will be glacial unless employers (and unions) pay more attention to the evidence on what actually works.

The data doesn’t lie. Progress stalled last year. In the English NHS the relative likelihood of White shortlisted candidates being appointed remained exactly 1.61 times more likely than that of shortlisted Black and Minority Ethnic (BME) candidates. Were that to apply throughout career progression a BME Band 5 nurse would be 28 times less likely to be a Director of Nursing on Band Very Senior Manager (VSM) than his or her white equivalent (1).

A large majority of these Action Plans are likely to fail for two main reasons. Firstly, they focus primarily on the interview process and not the rest of the recruitment and career progression cycle. Secondly, what they propose for the interview process itself simply ignores the evidence on what constitutes a fair interview process.

Who is recruited and promoted is affected by six main factors:

Firstly, social factors disadvantage applicants on the basis of their wider opportunities in life, particularly their school and university opportunities and experience which are in turn greatly influenced by class, race, gender, and disability in particular. I recall being told by a very senior NHS England leader that “we all know that Russell Group universities have brighter students” followed by group embarrassment when I said I didn’t go to one. The social capital (exacerbated by decades of positive discrimination) that some social groups gain gives them a serious edge when it comes to the “tap on the shoulder” for jobs.

Secondly, the way in which jobs and person specifications are designed and then advertised is prone to bias. Ridiculous job descriptions may deter women and BME applicants from applying but not bother men. The use of (pointless) “desirable” criteria facilitates affinity bias. Macho language in ads deters women (and probably BME) applicants.

Thirdly, the encouragement and stretch opportunities that some groups of staff get are much greater than those of others – the “tap on the shoulder” is widespread. Research suggests that key to career progression is access to “stretch opportunities” such as acting up, secondments and involvement in project teams. As NHS Improvement’s research put it “According to research, senior executives report their sources of key development as learning from experience in role and on the job (70%), learning from others, especially mentors, coaches and learning sets (20%), and formal coursework and training (10%)” (2).

I am repeatedly told by good managers how, when they personally encourage individual members of under-represented staff groups to apply for posts, such individuals often report this is the first time a senior manager has ever encouraged them to apply for a promotion. It is no wonder imposter syndrome is widespread and good staff persuade themselves they couldn’t do a potential promotion or see no point in being appointed as they believe the job already has someone’s name on it.

Fourthly, the shortlisting process is prone to bias. Poorly designed person specifications use flawed understanding of merit and disproportionately use past achievement not future potential as a key criteria. Elements of the application process such as a CV or a reference are seriously unreliable as a guide to future ability and prone to affinity bias. Such concerns are likely to have been compounded where there has been poor feedback (“ you were good but on the day someone else was better”) or pointless performance appraisals (3).

Fifthly, extensive scrutiny of local NHS action plans suggest that they focus excessively on some aspects of the interview process itself without any indication they are aware of what research suggests would be effective. Many NHS Action Plans place great emphasis on having a diverse panel and effective training for panel members despite little evidence that these two measures in isolation will make a decisive contribution to more diverse appointments. Whilst training for panel members may improve cognitive understanding, there is little evidence it makes a significant impact on decision making. The requirement for diverse panels is welcome but its impact is marginal.

There is insufficient understanding of the likelihood of bias within the actual interviews and the way specific processes exacerbate this.

There is far less emphasis on embedding the measures that we know are more likely to make a difference. What does the evidence suggest?

Meta analyses demonstrate that structured interviews with a scoring matrix that set out what poor, fair, good and outstanding answers to each question look like, assurance that the scoring is kept to and with more than one data point to draw on (such as a situational judgement test or work sample) make a difference (4). There is also good evidence that batch recruitment can improve diversity and reduce affinity bias, especially if it removes the future line manager from the final decision (5).

There is a wealth of research that could be used to improve the interview process itself. We know that first impressions of candidates and deference to the chair when panels score are sources of bias. We know that a diverse team that is inclusive and collaborative is more effective than a homogenous one that is not – and that diverse representation without inclusion is not sustainable. We know that positive action (support, encouragement, mentoring, coaching) can be helpful – but without addressing institutional bias is doomed to disappoint.

Finally, and above all, organisations need to take two key steps. Firstly emphasise debiasing their entire recruitment process rather than relying on training to debias panels. Secondly, insert accountability at every stage but especially in performance reviews, access to stretch opportunities, outcomes of interview processes, and outcomes at department, division, site, occupation. This requires Boards that hold themselves to account and model the behaviours they expect of others, understanding that what leaders do is crucial, and that such steps are about improvement not compliance.

How to do all this? Some NHS organisations are starting to adopt these principles, often influenced by the comprehensive evidence that demonstrates tick boxes are doomed to failure but evidenced interventions are not.

References

  1. https://www.england.nhs.uk/wp-content/uploads/2022/04/Workforce-Race-Equality-Standard-report-2021-.pdf (The data is from WRES 2021 report and the calculation is in No More Tick Boxes page 24)
  2. Developing People: Improving Care (2016)
  3. For a summary of the evidence see No More Tick Boxes pages 91-94
  4. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.172.1733&rep=rep1&type=pdf see also Kahneman D: Noise 2021 Chapter 24.
  5. Iris Bohnet. What Works: Gender Equality by Design 2016

Roger Kline

Roger Kline is Research Fellow at Middlesex University Business School. He authored No more tick boxes: a review of the evidence on how to make recruitment and career progression fairer and “The Snowy White Peaks of the NHS” (2014), designed the Workforce Race Equality Standard (WRES) and was then appointed as the joint national director of the WRES team 2015-17. Recent publications include the report Fair to Refer (2019) to the General Medical Council on the disproportionate referrals of some groups of doctors (co-authored with Dr Doyin Atewologun) and The Price of Fear (2018), the first detailed estimate of the cost of bullying in the NHS, co-authored with Prof Duncan Lewis.

Declaration of interests

I have read and understood the BMJ Group policy on declaration of interests and declare the following interests: none.

 

 

 

 

(Visited 2,658 times, 1 visits today)