By Eva Maria Hille, Patrik Hummel and Matthias Braun.

It takes its first steps over time, some of which are successful, some which aren’t. Its steps are becoming more regular and steady until the tempo picks up and it strives to run. As time goes by, it’s outgrowing its infancy. Similar to the steps of an infant, over the last decade, the concept of “Meaningful Human Control” passed through different levels in the discussion on autonomous weapon systems. Until today, the concept has been extensively discussed, and has even made it to the big stage: The European Parliament passed a resolution for banning autonomous weapon systems without Meaningful Human Control in 2018. Since 2015, the First Committee on Disarmament and International Security of the United Nations has devoted itself to the ongoing discussions on the concept of Meaningful Human Control within its side event series and panel discussions. Furthermore, associations like the UK-NGO ‘Article 36’ argued in 2015 for applying Meaningful Human Control for every single attack.

But what is behind the concept of Meaningful Human Control? First of all, a look into its application fields helps: It is discussed in the context of automated systems that act autonomously to a certain degree. These systems can make processes faster, more efficient, or better in other ways. However, when these kinds of systems are applied in areas where decision-making is a matter of life and death or could result in severe health damages, the question of who can control these systems or intervene in case of unintended consequences arises. To ensure the possibility of control and intervention, the concept of Meaningful Human Control was introduced.

What exactly does the concept of Meaningful Human Control entail? The researchers Filippo Santoni de Sio and Jeroen van den Hoven named two necessary conditions for an autonomous system to stay under control: The tracing condition says that the system’s design must enable it to trace back the results to humans taking part in the design or operation process. The tracking condition refers to the responsiveness of the system – it should be able to act in accordance with the moral reasons of those involved.

To avoid unintended consequences and harms, without having to completely forgo automated systems, Meaningful Human Control is applied in the health sector. In this field, more and more automated systems are being used, especially for diagnostics, treatment recommendations, surgery, care, telemedicine, public health, and many others. How exactly Meaningful Human Control can be deployed in these fields has not been researched yet. The concept of Meaningful Human Control in health is still in its infancy. Nevertheless, there are first approaches and ideas: Some of them deal with the concept superficially, while a few others have thoroughly thought through its application for a specific area of use. We have examined this status in our systematic review.

During our research, we came across some important findings. Compared to the context of defence and warfare, the health sector presents some specifics. On one hand, a variety of actors come into play: not only the often-mentioned physicians and designers but also the medical staff in general, as well as patients, their relatives, and others. All of them have their individual values and perceptions, resulting in the emergence of different needs and treatment options, which makes the area of health extremely complex. Against this background, there seems to be a theoretical gap regarding the concept of Meaningful Human Control in the health sector. Nonetheless, it must be stressed that Meaningful Human Control should not become just a label applied like a sticker to a health domain or a specific technical application without any theoretical consideration and resulting practical consequences. This could lead to ‘MHC-washing’, similar to the common phenomenon of ‘green-washing’.

Whether the concept of Meaningful Human Control can go beyond its initial steps in health or even fully outgrow its infancy depends on how successfully this complexity is considered and how well pitfalls are avoided. As a small contribution to that purpose, we propose a broader take on MHC along the three strands of enabling, enacting, and evaluating control.

We are positive that, if understood along these dimensions, the concept of Meaningful Human Control can help various stakeholders in the health sector leverage AI for better outcomes.

Paper title: Meaningful Human Control for Health? A Review

Authors: Eva Maria Hille¹, Patrik Hummel², Matthias Braun¹

Affiliations:

¹Chair of Social Ethics & Ethics of Technology, Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany

²Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology, Eindhoven, The Netherlands

Competing interests: None declared

Social media accounts of post author: Twitter @EvaMariaHille; @pat_humm @brau_matt