Bias, whether in data or algorithms, is a cardinal ethical concern in AI-CDSS. Data bias arises when data used to train the AI models are not representative of the entire patient population. This can lead to erroneous conclusions, misdiagnoses, and inappropriate treatment recommendations, disproportionately affecting underrepresented populations (Ganju et al., 2020). Model bias occurs when AI algorithms inherently favor certain outcomes or predictions over others due to their mathematical constructs. Such biases can compromise the fairness and effectiveness of AI-powered CDSS and perpetuate health disparities.

Yogarajan et al. investigates data and algorithmic bias in electronic health records (EHRs) in New Zealand. In response to the need to develop socially responsible and fair AI in healthcare for the New Zealand population, especially indigenous populations, the authors analyzed health data collected by clinicians to examine biases regarding data collection and model development using established techniques and fairness metrics. This study showed evident bias in the data and machine learning models employed in this study to predict preventable harm. The sources of bias may include missing data, small sample size and commonly available pre-trained embeddings to represent text data. This research underscores the crucial need to develop fair, socially responsible machine learning algorithms to enhance healthcare for underrepresented and indigenous populations, such as New Zealand's Māori.

AI models are often regarded as “black boxes” due to their complex and opaque decision-making processes. This opacity becomes ethically problematic when AI-CDSS are employed in healthcare. Clinicians and patients must understand the AI's predictions, including the inherent uncertainties, to make informed decisions. A lack of understanding of the inner workings of AI predictions also remains a key barrier to their responsible adoption in clinical workflows (Tonekaboni et al., 2019). However, AI models often lack transparency in communicating these uncertainties, which can impede trust and appropriate use of these systems.

Prabhudesai et al. address the challenge of quantifying and communicating uncertainty in Deep Neural Networks (DNNs) used for medical image segmentation, specifically in brain tumor segmentation. While DNNs provide accurate predictions, they lack transparency in conveying uncertainty, which can lead to false impressions of reliability and potential harm in patient care. The authors propose a computationally-efficient approach called partially Bayesian neural networks (pBNN), which performs Bayesian inference on a strategically selected layer of the DNN to approximate uncertainty. They demonstrate the effectiveness of pBNN in capturing uncertainty for a large U-Net model and showcase its potential for clinicians to interpret and understand the model's behavior. The methodology proposed by the authors holds promise of empowering clinicians in their interaction with AI-based CDSS and facilitating safer and more responsible integration of AI-CDSS in clinical workflows.

A substantial body of research has focused on developing innovative algorithms to enhance the technical performance of AI-CDSS (Alloghani et al., 2019; Barragán-Montero et al., 2021). However, relying solely on technological advancements is inadequate to ensure the successful implementation and user adoption of AI-CDSS. Recent studies have emphasized the significance of investigating human, social, and contextual factors that play a crucial role in the adoption of AI-CDSS (He et al., 2019; Schoonderwoerd et al., 2021). Consequently, there is a growing interest in the human-centered design of AI-CDSS and the exploration of fairness and transparency in AI. Therefore, it is imperative to synthesize the knowledge and experiences reported in this research area to shed light on future investigations.

Wang et al.'s systematic review effectively addresses this research gap. Their article provides valuable insights into the methodologies and tools employed for evaluating AI-CDSS, which can greatly benefit researchers. Furthermore, the review identifies various challenges associated with implementing AI-CDSS interventions, including workflow misalignment, attitudinal, informational, and environmental barriers, as well as usability issues. These challenges underscore the importance of examining and addressing sociotechnical obstacles in the implementation of AI-CDSS. The article also discusses several future research directions and design implications that can guide upcoming studies.

New system implementation in healthcare institutions is often accompanied by a change in clinical workflow and organizational culture (Zhang et al., 2019). Despite numerous efforts in advancing clinical decision support tools, most of these tools have failed in practice. Empirical research has diagnosed poor contextual fit as the cause, such as a lack of consideration of clinicians' workflow and the collaborative nature of clinical work (Wears and Berg, 2005). Thus, foundational research is needed to understand and improve expert work in an age of AI-assisted work, by integrating the richness of context and redefining the role of AI technology in clinical practice.

The paper by Ulloa et al. addresses the invisible labor involved in the integration of medical AI tools in healthcare. Through three case studies, the authors identify four types of labor: data labeling with clinical expertise, identifying algorithmic errors, translating output to patient care decisions, and fostering awareness of AI use. The authors highlight the need for standardized methodologies, reducing clinician burden, formalizing translation processes, and establishing social transparency to foster the adoption and integration of medical AI tools. Integration into existing workflows, usability, documentation, and ethical considerations are also crucial. The authors call for improved documentation of labor, workflows, and team structures to inform future implementations and prevent replicated efforts. They highlight the significance of recognizing and valuing the invisible labor involved in AI development and its impact on system implementation and society as a whole. The paper contributes to understanding the challenges and requirements associated with implementing AI in healthcare, emphasizing the need for a comprehensive approach that considers the labor and sociotechnical aspects to ensure successful and ethical adoption of medical AI tools.

As AI-powered CDSS herald a new era in healthcare, they bring along significant ethical issues that require urgent attention. Bias in data and models, lack of transparency, challenges in integration, and the complexities in evaluation present critical hurdles in harnessing the full potential of AI in healthcare. The four articles in this Research Topic have attempted to address these issues. Addressing these issues is crucial to ensuring that AI-powered CDSS are used responsibly and ethically, upholding the principles of fairness, transparency, and patient-centered care. As we continue to embrace AI's promise, it is essential that we also confront its ethical and contextual complexities, crafting an AI-infused future that is not just technologically advanced, but also user-centered and ethically sound.

RZ: Writing—original draft, Writing—review and editing. ZZ: Writing—review and editing. DW: Writing—review and editing. ZL: Writing—review and editing.