4 Mar, 13 | by BMJ Group
On Monday, a friend posted a picture of an enormous, triple layered chocolate cake on Facebook. It was, she wrote, a present for her boyfriend—a final year medical student—to celebrate his being awarded his top choice of location for his first job next year. Getting the first choice has meaning beyond simple preference: it enables the couple to spend the next two years living in the same part of the country.
But the elation and the chocolate cake were soon followed by sadness and anger. As you will have heard by now, the job allocation process is to be rerun, and, like thousands of other final year medical students, my friend’s boyfriend has no idea whether he will achieve the tantalisingly glimpsed first choice job again.
The limbo medical students are stuck in is down to an error—or perhaps more than one—in the machine marking of the “SJT” (Situational Judgement Test). This multiple choice exam was sat by final year medical students in December or January and forms a large chunk of the score that determines whether students achieve their first choice of location and job.
The latest news from the Foundation Programme Office (FPO—the body that administrates the application process) is that all answer sheets—previously machine marked—are to be checked by hand, to weed out scanning errors. (This task is being offloaded to medical schools, presumably because the FPO would buckle under its weight. Despite concerns over fairness, the FPO promises that medical schools will be involved only in verifying answer selection, and not rescoring). Students will learn next Friday whether they will still be working in the deanery they had been promised.
Unsurprisingly, students are miserable. “On Monday morning I found out I got my ideal deanery; I emailed my family and friends and celebrated the beginning of a new chapter in my life. On Tuesday that job offer was taken away, initially without apology,” said Sean Morris, a medical student at Bristol University. “Initially it took two months to mark the papers, and it’s taken less than two days for the allocation process to collapse.” Perceived poor communication by the FPO has come in for criticism—there is a sense that students were drip-fed revelations. “With the slow leakage of information, there is a feeling that we have been deceived,” said Clare Smyth, a student at the University of Glasgow.
The blunder comes soon after the FPO declared that the first Situational Judgement Test had been a “success,” pointing to its smooth roll-out. That press release has quietly vanished from the FPO’s website. It seems it is not just medical students who gloated about their victories too early.
For many, this scandal will sound strong echoes of the 2007 MTAS fiasco (or, rather, fiascos), in which a poorly executed switch to a centralised computer system of job application left many doctors without training posts. Although the FPO promise that the gaffe will not prevent junior doctors taking up jobs in August, the cries of demoralisation and abandonment are similar. The threat always made at these times is that junior doctors will drop out of training to pursue alternative careers or jobs abroad. Will they? I would guess not. Despite the threats, medical students are almost powerless in this situation: the FPO operates a monopoly. Barring the few who are tempted by migration to Australia or New Zealand, doctors must stay with the system.
This is unfortunate, for, despite the impression we might get from the FPO language of “first choices”— that the allocation of first preference is a luxury—the awarded deanery has huge implications for many students. The image of medical school leavers as fresh faced and footloose 23 year olds is outdated and unrealistic: many have partners, children, or financial ties that mean they are desperate to be allocated to a particular deanery; hence the huge stress.
How did the marking error occur? It’s difficult to unpick. For starters, there seems to have been more than one problem. In the week before the results were released, the FPO say they noticed than an answer sheet had been creased and this had caused the scanner to give the applicant the wrong mark. This prompted a check of every single answer sheet, which threw up another potential inaccuracy. And the checking of this led to the identification of—guessed it?—yet another error. By this time, results had already been released to students.
This catalogue of ineptitude will send a chill down the spine of anyone who has ever had an important exam hinge on machine marking—which, in the younger generation of medics, is just about everyone. (Situational Judgement tests are also used to place GP trainees.) Is the reliability of most machine marked exams similarly shoddy? The FPO account gives the impression that, the more checking was done, the more mistakes were found. Has perceived reliability to date hung on a deficit of checking?
Unsurprisingly, the BMA has reacted fiercely to the situation. In particular, it is demanding that any student who has put money down after hearing about their initial allocation should not lose out. “It would not be acceptable for any student to lose out financially through an unexpected change in their foundation programme placement, and we would expect the UKFPO to compensate them accordingly.” Call me cynical, but this feels like pie in the sky. I don’t expect the FPO to be handing out cash any time soon.
The problem—as with any situation like this—is that, in the absence of hard facts, speculation is rife. The North East Thames Foundation School has encouraged students who were initially allocated to it to attend its open day: “Despite the uncertainties that the situation has raised, and the possibility that applicants may still be allocated elsewhere if the allocation is re-run, we believe that there will be minimal impact on foundation school allocations.” Is the school right? On the one hand, the scanning error probably affected small numbers of papers, and so allocations might barely change. On the other hand, it seems that most students have test scores that are closely clustered together, and so even a small change to some scores might have a knock-on effect on many allocations.
Even if the new allocation doesn’t snatch away scores of already awarded jobs, the bitterness this situation has caused among a new cohort of doctors will last. Neil Chanchlani, a medical student at the University of Birmingham, reports that: “In an email, my medical school advised us to ‘to be philosophical and concentrate on work and the other things you’ve got going on. These are one of those situations in life that tests your resilience.’ I’d like them to be offered a job, have it taken away, and as a remedy, tell them to ‘be philosophical.’”
Isobel Weinberg is the editor, Student BMJ.