Google is in hot water. First of all, the artificial intelligence (AI) focused branch of the organization, Google DeepMind, recently held a public meeting on the hot topic of accessing NHS patient information.

Google already has access to 1.6 million patient records, and plans to build an electronic portal that allows patients and doctors to track full medical histories in chronological order via an app on a smartphone. This did not go down well with some of the attendees, who expressed concerns about consent, data ownership, and the need for individuals to opt out if they do not want Google using their data.

Meanwhile, another Google offshoot, Google Brain, has apparently uploaded the content from around 11 000 novels into a neural network, to “capture the nuance of language better” in an attempt to improve communication between humans and AI systems. According to the report, using novels is particularly useful as they contain “language that frequently repeats the same ideas, so the AI system could learn many ways to say the same thing.” The only problem is that Google didn’t ask permission to use the works of fiction—the novels were taken from a collection of free books written by unpublished authors.

To me, this effort by Google to better understand language and communication between machines and people should be applauded, especially as the tech behemoth is increasingly interested in healthcare. For an AI to work efficiently and effectively in that setting, it will need to understand the nuances of the language of healthcare from the patient perspective, and not simply the jargon favored by clinicians.

Potential confusion could arise with homophones (words that sound the same, but which have different meanings and spellings, such as cabbage and CABG) and homographs (words that are spelt the same, but which have different meanings, for example, one man’s emergency department (ED) is another’s erectile dysfunction). And that’s before we even consider dialectal differences across the UK and other countries—in Scotland what would an AI system make of bampots, bevvies, and bairns, or the use of a stookey for a broken arm?

There is already abundant evidence that many patients encounter barriers to understanding health related information, and that materials and other content created by clinicians often fail in terms of readability. Ideally, if AI figures in the future of healthcare, it might help to ease these communication problems, but at the very least developers should be careful not to add to them.

It’s also worth noting that AI development itself has highlighted the under-recognized clinical challenge of patients and doctors’ different understanding of what is being said—a problem exemplified by the “Chinese room argument.”

Here a native English speaker is in a room. Through a window, a Chinese person shows him Chinese lettering. The English speaker has a manual in English, instructing him how to show lettering specific for the lettering he sees through the window. He is not aware that the image he sees is a question, and that what he then shows is the correct answer. The Chinese person outside cannot work out that the English speaker does not understand Chinese. The corollary in healthcare is that a clinician may impart information that the patient is able to repeat back to them, but unless formally tested, it remains unknown as to whether the patient understood the meaning or implication of this information and its relevance to them.

So, for technology companies such as Google to create useful artificial intelligence systems that benefit patients, they will need to collect patient relevant language. To do that, such companies are likely to need to access language from a variety of sources, including handwritten notes, letters, emails (i.e. medical records), and presumably (and controversially) also listen directly to patients talking with their clinicians.

Given the potential positive impact of AI in healthcare, this is probably a step worth taking. For some though, and to paraphrase Ronald Reagan, the most terrifying words in the English language when it comes to accessing healthcare could be “I’m from Google and I’m here to help . . . ”

David Kerr wears many hats—physician; editor of Diabetes Digest; and founder of DiabetesTravel.org, a free service for travellers with diabetes, and Excarbs.com, which focuses on exercise and insulin. He is director of research and innovation at the William Sansum Diabetes Center in Santa Barbara, California, and mHealth lead for the Diabetes Technology Society (unpaid). You can follow him on Twitter (@GoDiabetesMD).

Competing interests: The author has no relevant competing interests to declare.