By Robert Ranisch and Sabine Salloch
If AI ever poses a real existential threat to humanity, these may be the days when the broader public becomes aware of it. Following a series of incidents involving increasingly autonomous AI systems, leading figures at several so-called frontier labs have themselves called for slowing the pace of AI development. At the same time, media outlets have been asking some version of the same question: “Will AI really kill everyone?” The headlines have been full of gloomy scenarios involving increasingly autonomous AI systems. And, at least in part, there seems to be a certain guilty pleasure in imagining the various ways in which AI might go rogue and wipe out humanity.
One reason why these old fears of losing control over technology suddenly feel more tangible is the recent progress of agentic AI: (semi-)autonomous AI systems that can perform tasks on their “own” – from conducting research to exploiting vulnerabilities in computer systems. While the “year of agentic AI” was already announced in 2025, now seems to be the showtime for some of these developments.
Of course, there are good reasons to be sceptical about this recurring turn from AI boom to AI doom. Some have argued that debates about existential AI risk are little more than familiar marketing: a manoeuvre by Big Tech to distract from more immediate harms, or even a form of “criti-hype” that ultimately fuels excitement around frontier models. Doomsday warnings can also sit rather comfortably alongside the commercial interests of the companies developing these systems: who, after all, is better positioned to save us from dangerous AI than the very companies building it? The warnings might be correct. We don’t know. But we are pretty sure that before AI agents eventually kill us all, they will have more mundane yet potentially profound consequences across many sectors of society, including healthcare.
Around the same time that AI agents became the next big thing in tech, digital-health evangelists announced a coming “new era of agentic medicine”: systems that function as healthcare teams or patient assistants, performing specific tasks with increasing autonomy. Recent studies such as this or this already give us an idea of what this might look like: AI systems that do not merely provide answers but also collect information, reason about diagnoses and treatment, and actively coordinate several steps of care.
When computer systems begin to act more autonomously in the medical realm, this obviously provokes various ethical questions. Are they safe? Who is accountable? How much human oversight is needed? At the same time, they may change the way doctors and patients interact with each other and with machines. And this is the starting point of our Extended Essay, recently published in the Journal of Medical Ethics.
Many of the ethical questions surrounding agentic AI actually resemble debates we already know from LLMs in healthcare: safety, privacy, bias, accountability, transparency, and so on. What seems particularly interesting to us is something else: the possible impact of this technology on the very moral fabric of medicine.
To explore this, we look at agentic AI through the lens of techno-moral change. The basic idea is quite simple: technologies do not only create new possibilities. They can also contribute to a transformation of established norms and values. Take airplanes: Once you can cross a continent within a few hours, living far away is no longer quite the excuse it used to be for missing your grandmother’s birthday. Or take birth control: The contraceptive pill was not simply another pharmacological technology. Giving women greater control over reproduction contributed to changes in sexual morality and social relationships.
Something similar could happen with agentic AI: What if AI agents could guide patients through complex diagnostic procedures better than physicians? Could physicians eventually have an obligation to let the agentic network do their job? What if patients prefer the constant availability and “empathic” communication of a machine to interactions with their GP? And what happens to the patient-physician relationship when it is no longer really a relationship between two people, but part of a network involving physicians, patients, and multiple interacting artificial agents?
Such developments could have consequences for medical morality: for how doctors and patients interact and for where epistemic authority is located. Suddenly, certain tasks that have so far been closely associated with the role of the human physician might be considered better handled by machines. Simulated empathy might, for some patients, become as valuable as human empathy. And what we currently regard as a good patient-physician relationship might be open to alternatives, with artificial partners becoming part of the game.
This brings us to a deeper philosophical problem. We all know that morality has changed before and will change again. But if technological developments help drive such changes, it seems worth discussing what we should actually aim to preserve and what we might happily leave behind. Despite moral progress in healthcare, there is no “Archimedean point” in the history of medicine from which we can unequivocally derive the normative standards of the patient-physician relationship. But this does not mean that technological change should simply be left to run its course. Rather, it makes reflection on the values and purposes we have good reasons to preserve all the more important. Such reflection may, in turn, inform design choices or limits for new technologies, such as agentic AI.
Essentially, that is the bottom line of our essay, although we can provide little more than a prelude. If agentic AI becomes a substantial part of healthcare, we should not only ask whether these systems are safe and effective. We should also ask what kind of medicine they are helping to create, what we hold dear in the patient-physician relationship, and whether tomorrow’s medical morality still contains those things we consider worth preserving.
Paper Title: Agentic AI, medical morality and the transformation of the patient–physician relationship
Authors: Robert Ranisch (1) and Sabine Salloch (2)
Affiliations:
- Faculty of Health Sciences, University of Potsdam, Potsdam, Germany; Hasso Plattner Institute, University of Potsdam, Potsdam, Germany
- Institute for Ethics, History and Philosophy of Medicine, Hannover Medical School, Hanover, Germany
Competing Interests: None to declare.
Social Media Accounts:
Robert Ranisch: LinkedIn, Bluesky: @robranisch.bsky.social
Sabine Salloch: Bluesky: @salloch.bsky.social