Giles Maskell: Radiology without context becomes a game of “Where’s Wally”

When it comes to image interpretation, context can be all-important

A physician friend posted a photo on our group chat with the caption: “Where am I?” The photo showed a flower bed, backed by a stone wall. There were no clues. There must be thousands of similar scenes throughout the country, and for that matter in other countries. So I guessed, and I happened to guess correctly. My friend was impressed: “How on earth did you do that?” It certainly wasn’t because I recognized the flowers or the wall. I have little or no interest in gardening. My guess was based not on the elements within the photo, but largely on what I know about my friend. I won’t go too deeply into this for obvious reasons, but I know roughly where she lives, how she spends her time and a certain amount about her social and cultural heritage and preferences. From all that I could have a guess at where she might be visiting, and what destination she might choose to represent with this particular photo.

That’s a rather long-winded way of pointing out that when it comes to image interpretation, context can be all-important, and of course the same applies in radiology. Pattern recognition is important, but is usually not enough on its own. Accurate interpretation relies on a combination of pattern recognition and the application of contextual information. Radiology without context becomes a game of “Where’s Wally”, albeit a version of the game with rather perverse rules in which Wally is absent most of the time, often wears different outfits and also lends his stripy jumper to other individuals whose identification is of no interest or value.

Finding the right balance between the two elements of pattern recognition and context can be very challenging. There will be times when the two pull in different directions—it has all the features of “x”, but is that really likely in this particular patient? Or alternatively, patients presenting like this usually turn out to have “y”, but the images look nothing like that.

This has implications for the application of Artificial Intelligence to radiology. For a computer to interpret the image described above, it could either employ more sophisticated pattern recognition—the precise shade of the delphiniums or the arrangement of bricks in the wall might carry clues which the ordinary human observer would miss—or alternatively it could factor in contextual information. My strong suspicion is that it will need to do both.

Computers will certainly make use of better pattern recognition—quantification and textural analysis, for example, will extract information which is invisible to the human eye from standard Computed Tomography (CT) and Magnetic Resonance (MR) images. But it is in the application of contextual information, and crucially in balancing the relative importance of that information in a particular individual against the importance of features contained within the images themselves, that the real gains may be made.

Radiologists make these judgments all the time and, it has to be said, frequently come up with the wrong answer. An optimistic view of the future is that computers will help us to get it right more often.

This, by the way, may help to explain why radiologists can get so fractious when presented with less than the full and accurate clinical picture. Nobody wants to be a Wally-ologist.

Giles Maskell is a radiologist in Truro. He is past president of the Royal College of Radiologists.

Competing interests: None declared. 

  • Anton Joseph

    Without context, radiology is a game of Where’s Wally?
    Let us put our house in order
    Giles Maskell is absolutely right to highlight the game that radiologists are asked to play several times a day. The lack of adequate history or clinical information often leaves the radiologists making an intelligent guess of the diagnosis using pattern recognition based on their experience and any contextual information they might gather. There is however no mention in the opinion piece of what might be labelled self imposed suppression of highly relevant information by the radiologists themselves.
    Most investigations carried out in the radiology department, be it a simple radiograph, CT scan, MRI or ultrasound is an attempt to image and display the anatomy or pathological changes. Would knowledge of the normal anatomy or the type of abnormality not make a very significant contribution to the ‘contextual’ information?
    From the design of the course with several hours devoted to physics and hardly any for pathology for the training of radiologists does it not imply that the knowledge of interaction of radiation with matter or the physics of image production makes a greater contribution to the interpretation of the image than the pathology?
    Entering radiology from forensic medicine with a moderate background in pathology and a moderate physics background from radiation biology it was most disappointing to hear that pathology was dropped from the curriculum for the fellowship examination in radiology. Rumour has it that it was due to the lack of competent radiologists to teach or examine in pathology, a naive explanation in view of the fact that physicists were invited to teach and examine in the fellowship examination and support the radiologists.
    Another lame excuse in my opinion is the lack of time to fit in pathology into the curriculum. Knowledge of pathology is certain to contribute to the interpretation of the image more than the knowledge of the physics in making the diagnosis. It is time that the Royal College of Radiologists create a common sense approach to the time allocated to physics and pathology in the training of radiologists.
    Anton E Joseph
    Retired Consultant radiologist
    Wallington, Surrey.