{"id":4591,"date":"2024-09-19T11:38:41","date_gmt":"2024-09-19T10:38:41","guid":{"rendered":"https:\/\/blogs.bmj.com\/medical-ethics\/?p=4591"},"modified":"2024-09-19T11:38:41","modified_gmt":"2024-09-19T10:38:41","slug":"could-xai-harm-patients-in-time-sensitive-environments","status":"publish","type":"post","link":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/","title":{"rendered":"Could XAI harm patients in time-sensitive environments?"},"content":{"rendered":"

By Andreas Wabro.<\/p>\n

Achieving transparency and interpretability of algorithmic predictions remains an important research goal for many AI experts around the world. In particular, the epistemic benefits of explainable AI (XAI) methods have been widely discussed, and especially in the context of healthcare, international institutions and academic experts often call for measures to improve physicians’ understanding of algorithmic output as far as technically feasible. Many experts, with very valid arguments, proclaim various potentials of such explainable AI methods, particularly in the field of medicine.<\/p>\n

But what can be expected from XAI methods incorporated into the system design of AI-based decision support systems when such methods are applied in time-sensitive clinical environments?<\/p>\n

Drawing on expert interviews we conducted with software engineers working in the field of computational cardiology, we sought to elaborate on the most prevalent ethical implications arising from XAI usage in such demanding situations. And while this particular question seemed trivial to us at first, it soon proved to be of far-reaching importance and raised striking ethical implications once we decided to dig deeper.<\/p>\n

After conducting a detailed ethical analysis, we found that time-sensitive clinical environments may indeed conflict with the use of XAI for a number of reasons, which we discuss in our recent JME paper<\/a>. Importantly, because time-sensitive environments typically require immediate medical attention or urgent surgical resolution, they often do not allow the physicians or surgeons responsible for making the clinical decision at hand to conduct a detailed analysis of the AI explanations provided.<\/p>\n

However, in-depth analysis is often necessary to rule out potential XAI fallacies or intrinsic errors, which are difficult to detect with limited exposure time. In extreme cases, where the clinical environment prevents doctors or surgeons from adequately considering the system’s explanations, XAI methods could even harm patients’ physical integrity.<\/p>\n

In particular, as XAI methods still appear to be prone to various errors, they continue to present an intrinsic set of challenges that are less well suited to being dealt with in time-critical clinical situations. In such scenarios, a well-designed black-box AI might, after weighing conflicting arguments, be more ethically desirable than a less well-designed XAI that fails to live up to its promises and the heightened expectations that flow from its specific nature and design.<\/p>\n

 <\/p>\n

\n

Paper title:<\/strong> When time is of the essence: ethical reconsideration of XAI in time-sensitive environments<\/a><\/p>\n

Authors<\/strong>: Andreas Wabro, Markus Herrmann, Eva C Winkler<\/p>\n

Affiliations:<\/strong> National Center for Tumor Diseases (NCT) Heidelberg, NCT Heidelberg, a partnership between DKFZ and Heidelberg University Hospital, Germany, Heidelberg University, Medical Faculty Heidelberg, Heidelberg University Hospital, Department of Medical Oncology, Section Translational Medical Ethics<\/p>\n

Competing interests<\/strong>: None declared.<\/p>\n

Social media accounts of post author<\/strong>:\u00a0 @EthicsWinkler<\/a>; @ethicswinkler.bsky.social<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

By Andreas Wabro. Achieving transparency and interpretability of algorithmic predictions remains an important research goal for many AI experts around the world. In particular, the epistemic benefits of explainable AI (XAI) methods have been widely discussed, and especially in the context of healthcare, international institutions and academic experts often call for measures to improve physicians’ […]<\/p>\n

Read More…<\/a><\/p>\n","protected":false},"author":354,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8068,8057],"tags":[],"class_list":["post-4591","post","type-post","status-publish","format-standard","hentry","category-artificial-intelligence","category-medical-ethics"],"yoast_head":"\nCould XAI harm patients in time-sensitive environments? - Journal of Medical Ethics blog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Could XAI harm patients in time-sensitive environments? - Journal of Medical Ethics blog\" \/>\n<meta property=\"og:description\" content=\"By Andreas Wabro. Achieving transparency and interpretability of algorithmic predictions remains an important research goal for many AI experts around the world. In particular, the epistemic benefits of explainable AI (XAI) methods have been widely discussed, and especially in the context of healthcare, international institutions and academic experts often call for measures to improve physicians’ [...]Read More...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/\" \/>\n<meta property=\"og:site_name\" content=\"Journal of Medical Ethics blog\" \/>\n<meta property=\"article:published_time\" content=\"2024-09-19T10:38:41+00:00\" \/>\n<meta name=\"author\" content=\"Hazem Zohny\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Hazem Zohny\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/\"},\"author\":{\"name\":\"Hazem Zohny\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#\\\/schema\\\/person\\\/e73152f6aa4e164c7d625d77cf4fed35\"},\"headline\":\"Could XAI harm patients in time-sensitive environments?\",\"datePublished\":\"2024-09-19T10:38:41+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/\"},\"wordCount\":454,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#organization\"},\"articleSection\":[\"Artificial intelligence\",\"Medical ethics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/\",\"name\":\"Could XAI harm patients in time-sensitive environments? - Journal of Medical Ethics blog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#website\"},\"datePublished\":\"2024-09-19T10:38:41+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/2024\\\/09\\\/19\\\/could-xai-harm-patients-in-time-sensitive-environments\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Could XAI harm patients in time-sensitive environments?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#website\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/\",\"name\":\"Journal of Medical Ethics blog\",\"description\":\"A blog to discuss the ethics of medicine in its many guises and formats.\",\"publisher\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#organization\",\"name\":\"Journal of Medical Ethics blog\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/files\\\/2026\\\/04\\\/jme-logo.png\",\"contentUrl\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/files\\\/2026\\\/04\\\/jme-logo.png\",\"width\":200,\"height\":50,\"caption\":\"Journal of Medical Ethics blog\"},\"image\":{\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#\\\/schema\\\/logo\\\/image\\\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/#\\\/schema\\\/person\\\/e73152f6aa4e164c7d625d77cf4fed35\",\"name\":\"Hazem Zohny\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4d1bd322d5ff8e6f31a624003143903d0ba7346af8cdb1fd6b5e95924b92ece1?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4d1bd322d5ff8e6f31a624003143903d0ba7346af8cdb1fd6b5e95924b92ece1?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/4d1bd322d5ff8e6f31a624003143903d0ba7346af8cdb1fd6b5e95924b92ece1?s=96&d=mm&r=g\",\"caption\":\"Hazem Zohny\"},\"sameAs\":[\"https:\\\/\\\/www.practicalethics.ox.ac.uk\\\/people\\\/dr-hazem-zohny\"],\"url\":\"https:\\\/\\\/blogs.bmj.com\\\/medical-ethics\\\/author\\\/zohnyh\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Could XAI harm patients in time-sensitive environments? - Journal of Medical Ethics blog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/","og_locale":"en_US","og_type":"article","og_title":"Could XAI harm patients in time-sensitive environments? - Journal of Medical Ethics blog","og_description":"By Andreas Wabro. Achieving transparency and interpretability of algorithmic predictions remains an important research goal for many AI experts around the world. In particular, the epistemic benefits of explainable AI (XAI) methods have been widely discussed, and especially in the context of healthcare, international institutions and academic experts often call for measures to improve physicians’ [...]Read More...","og_url":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/","og_site_name":"Journal of Medical Ethics blog","article_published_time":"2024-09-19T10:38:41+00:00","author":"Hazem Zohny","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Hazem Zohny","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/#article","isPartOf":{"@id":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/"},"author":{"name":"Hazem Zohny","@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#\/schema\/person\/e73152f6aa4e164c7d625d77cf4fed35"},"headline":"Could XAI harm patients in time-sensitive environments?","datePublished":"2024-09-19T10:38:41+00:00","mainEntityOfPage":{"@id":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/"},"wordCount":454,"commentCount":0,"publisher":{"@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#organization"},"articleSection":["Artificial intelligence","Medical ethics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/","url":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/","name":"Could XAI harm patients in time-sensitive environments? - Journal of Medical Ethics blog","isPartOf":{"@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#website"},"datePublished":"2024-09-19T10:38:41+00:00","breadcrumb":{"@id":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/blogs.bmj.com\/medical-ethics\/2024\/09\/19\/could-xai-harm-patients-in-time-sensitive-environments\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/blogs.bmj.com\/medical-ethics\/"},{"@type":"ListItem","position":2,"name":"Could XAI harm patients in time-sensitive environments?"}]},{"@type":"WebSite","@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#website","url":"https:\/\/blogs.bmj.com\/medical-ethics\/","name":"Journal of Medical Ethics blog","description":"A blog to discuss the ethics of medicine in its many guises and formats.","publisher":{"@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/blogs.bmj.com\/medical-ethics\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#organization","name":"Journal of Medical Ethics blog","url":"https:\/\/blogs.bmj.com\/medical-ethics\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#\/schema\/logo\/image\/","url":"https:\/\/blogs.bmj.com\/medical-ethics\/files\/2026\/04\/jme-logo.png","contentUrl":"https:\/\/blogs.bmj.com\/medical-ethics\/files\/2026\/04\/jme-logo.png","width":200,"height":50,"caption":"Journal of Medical Ethics blog"},"image":{"@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/blogs.bmj.com\/medical-ethics\/#\/schema\/person\/e73152f6aa4e164c7d625d77cf4fed35","name":"Hazem Zohny","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/4d1bd322d5ff8e6f31a624003143903d0ba7346af8cdb1fd6b5e95924b92ece1?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/4d1bd322d5ff8e6f31a624003143903d0ba7346af8cdb1fd6b5e95924b92ece1?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/4d1bd322d5ff8e6f31a624003143903d0ba7346af8cdb1fd6b5e95924b92ece1?s=96&d=mm&r=g","caption":"Hazem Zohny"},"sameAs":["https:\/\/www.practicalethics.ox.ac.uk\/people\/dr-hazem-zohny"],"url":"https:\/\/blogs.bmj.com\/medical-ethics\/author\/zohnyh\/"}]}},"_links":{"self":[{"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/posts\/4591","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/users\/354"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/comments?post=4591"}],"version-history":[{"count":0,"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/posts\/4591\/revisions"}],"wp:attachment":[{"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/media?parent=4591"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/categories?post=4591"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.bmj.com\/medical-ethics\/wp-json\/wp\/v2\/tags?post=4591"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}