By Nicole Gross
Medical professionals seem to be stuck on the topic of whether or not AI is useful, robust and ethically-sound enough to become scaled more widely into clinical work and decisions. Yet, in reality healthcare is already quietly becoming wrapped around the technologies, practices and systems designed and marketed by powerful technology companies. AI is so much more than machine learning, algorithms, scribes, or chatbots – a technology, tool or ‘thing’’, so to speak. Instead, AI is a phenomena in-the-making that results in a series of complex and emergent practices and power structures, all of which carry ethical risks and implications, and these are already seeping in and through clinical work, patient interaction, healthcare organisation, and procurement.
To illustrate: Clinicians spend a lot of their time on direct patient care, including communication and making a diagnosis, but they spend even more time on indirect care, including planning and collaborating, reviewing evidence, and documenting practically everything they do. Driven by Big Tech’s rapid expansion into healthcare, AI is now increasingly taking over tasks such as recording and transcribing notes, summarizing issues, producing documentation, responding to queries, and even making diagnosis, or assisting with treatment decisions. This means that the clinician’s time is now increasingly occupied by reviewing, validating and correcting AI-generated outputs, and deciding whether AI advice should be ignored or not. However, this is where ethical principles, such as ‘do no harm’ can become murky: emerging evidence suggests that AI can steal the clinician’s (first) thought and wrongly anchor their judgment, act to impair the learning of trainee doctors, and make it increasingly difficult for clinicians to decide how much justification is actually needed to disagree with opaque AI outputs.
In terms of patient interactions: AI enables patients to become more active in their healthcare journey, which means that they might arrive at their doctor’s office with (possibly wrong) AI-generated knowledge and they might also try to corroborate what the doctors say against the outputs of AI. Such so-called ‘AI-powered patients’ may also choose to completely disregard what professionals recommend altogether, based on what they have learned from AI. While doctors ought to grant their patients autonomy, thus respect the patient’s right to make their own independent choices and decisions, they are also ethically required to act in the best interest of the patient and actively do good. In a move towards triadic care, clinicians will now have to deal with (perhaps even embrace) these AI-powered patients and work towards building health literacy and critical AI skills with them. This means asking important questions about what tools, technologies and platforms they have used, how and why, and help them to look beyond AI’s affirming, confident but often also erroneous answers. By engaging proactively and acting ethically, clinicians and medical professionals end up organising their work around an increasingly wide array of AI platforms, digital systems and algorithmic technologies here.
Finally, AI has also huge impact on the behind-the-scenes organisation of healthcare. For non-clinical decision-making, AI is already performing workflow automation, email management, scheduling, billing, and essential IT tasks. These are often bolstered by procurement decisions that clinicians have very little control over, but which have a significant impact over their daily practices and ethical decision-making, including those related to justice. Procurement includes which foundational models, AI scribes, EHR systems, clinical support systems, and workflow platforms exist in the hospital, and also how they are chosen, implemented and used. Even when AI is not used directly by the medical professional or clinician, these systems act to influence medical standards, document flows, data management, infrastructures, interoperability, workflows, and essential support systems. While procurement can shape the trajectory of innovation in a way that healthcare can be delivered more equitably (and at higher speed and lower cost too), it also changes what fairness in medical care looks like, what products or technologies define ‘good care’, how resources are allocated, what data is collected, how it is used and by whom, and which companies gain power and exert influence over healthcare delivery (e.g., Palantir’s contract with the UK’s National Healthcare System has opened the door for power to be abused).
AI belongs in healthcare – in 2026 that is no longer a question. However, what warrants further investigation and governance is when the logics of AI systems, designed and marketed by powerful technology companies, infuse healthcare to the point that core ethical principles like beneficence, non-maleficence, and justice become quietly distorted. While healthcare has arrived at this point already, it is worth remembering that AI is still in-the-making. Therefore, it is not too late to make investments, engage in management efforts and showcase political will to redirect fundamental logics back to healthcare’s commitments to ethics, professional values and public mission, rather than letting these follow the commercial, technical and socio-political assumptions that are so deeply embedded in AI.
Author: Nicole Gross
Affiliation: National College of Ireland
Conflicts of Interest: None to declare
Social media accounts: LinkedIn: https://www.linkedin.com/in/nicole-gross-a605b412 ; X: @tech_spaces