As the NHS explores the potential impact of AI on healthcare provision we should be wary of the revolving door between Big Tech companies and the NHS, says Paul Tiffin

At the start of the year, Juliet Bauer, chief digital officer for NHS England, announced that she was joining LIVI, an independent provider of remote primary care advice. Months earlier, Michael Macdonnell, NHS England’s national director of Transforming Health Systems, declared he was leaving his role to join DeepMind Health, a subsidiary of Google. When NHS leaders join the private sector they take with them valuable insights into how Big Tech can access Europe’s largest single health market. The traffic goes both ways. Nicola Blackwood, who lost her Oxford parliamentary seat in 2017, was awarded a peerage, permitting her to return to government as minister for innovation. While out of government Blackwood undertook paid work for health sector technology companies such as Eagle Genomics, specialising in big data analytics for the health sciences. 

Matt Hancock has publicly declared his enthusiasm for artificial intelligence (AI) based technology. An article in The Guardian suggested that politicians had been seduced by the positive media coverage of AI, often largely based on the industry’s own press releases. My own view is that it is more complicated than that. The idea that AI will revolutionise the delivery of healthcare plays well into the “personal responsibility” agenda of the current government and nicely distracts from the ongoing workforce crisis. Indeed, Juliet Bauer, while at NHS England, was quoted as saying, regarding health, “We need to move away from the paternalistic, clinician-led culture.” The (not so) latent narrative seems clear; you don’t need to worry about a shortage of mental health workers as you can always chat to “Woebot” about your troubles. We needn’t concern ourselves about cuts to public health as there will be a new, empowering app out soon to let you take control of your own wellbeing without interference from pesky “experts.”

All of the above could be dismissed as the paranoid ramblings of a Luddite, cynical NHS consultant (guilty on at least one of those charges). Except that I am also a data scientist, which is, apparently, now the de rigueur term for someone like me who spends much of their time wrangling recalcitrant data and building predictive models. I have been struck by the almost daily announcements of AI “microdiscoveries” in the news. Nevertheless, however technically impressive some of these predictive feats are in datasets, there have been almost no real world examples of where AI has either improved clinical outcomes or saved money. But we have been here before. Remember how telemedicine was going to revolutionise healthcare? Well it hasn’t. It is still, nonetheless, finding its place, and being implemented at a quieter, more sensible pace. AI-based systems hopefully will do likewise.

Many parallels between Big Tech and Big Pharma are emerging. As The BMJ itself has regularly highlighted, the influence of Big Pharma is pervasive at all levels of the health service. Rolled out rapidly at scale, and misapplied, algorithms—just like novel medications—can cause unintended harm. The mathematician, Cathy O’Neil, refers to them as potential “weapons of math destruction” in her book of the same title. Already concerns have been raised about the implementation of remote and automated consultation services, as well as issues regarding data privacy and sharing. As with Big Pharma, the government is keen to subsidise data science companies with tax payers’ money in the hope that the growth of this sector will fuel UK economic growth. However, in the case of biotechnology, it is unclear what the return for the taxpayer has been. 

We need to learn the lessons of our relationship with Big Pharma in order to manage, and build a trusting partnership with Big Tech. As with the former, this needs to be based on transparency, both of conflicts of interest and access to study data. Such relative transparency was hard won and an ongoing challenge. In the case of Big Tech, code is likely to be considered commercially sensitive. However, where queries arise independent experts should still be granted access to software. Machine learning studies are notoriously difficult to reproduce and clearer reporting standards are needed. Independent evaluation should focus on their potential for efficiencies, but also for harm, including any disadvantage to society’s “digitally excluded.” We need to work harder to retain expertise within the public sector, if only to collaborate meaningfully with industry, and expertly challenge them when concerns are raised.

Putting the public sector and Big Tech on an equal footing will be difficult. Firstly, we must begin by adopting a more realistic view of the potential for AI to improve health services. Secondly, we need to recognise the subtle but powerful corporate influences on policy related to healthcare technology. A scene from the 1999 movie “The Matrix” has given rise to a cultural meme. In it the protagonist Neo (Keanu Reeves) is offered a choice by his mentor, Morpheus; take a blue pill and stay in the comforting, but ultimately deceitful, virtual reality wonderland created by the machines, who have now enslaved mankind. Or swallow the red pill and wake up to the post-apocalyptic world as it really is, with Morpheus promising Neo to show him ‘…how deep the rabbit hole goes…’. In order to redress the current power imbalance between the private and public sectors perhaps we need to start by simply swallowing our own red pills… 

Paul A Tiffin, is NIHR Career Development Fellow, Reader in Psychometric Epidemiology, Honorary Consultant in the Psychiatry of Adolescence, University of York, Hull York Medical School. 

Disclaimer: PAT is supported in his research by an NIHR Career Development Fellowship. This paper presents independent research part-funded by the National Institute for Health Research (NIHR). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Competing Interests: PAT has previously received research funding from the ESRC, the EPSRC, the DH for England, the UKCAT Board, and the GMC. In addition, PAT has previously performed consultancy work on behalf of his employing University for the UKCAT Board and Work Psychology Group and has received travel and subsistence expenses for attendance at the UKCAT Research Group.