Early AI research in healthcare focused primarily on predictive accuracy, sensitivity, and specificity. As models became more complex and influential, concerns emerged regarding bias, explainability, and generalizability. In response, scholars and policymakers advanced ethical principles intended to constrain harmful applications of AI, often drawing on broader discussions of AI ethics across industries.

The WHO's guidance on the ethics and governance of AI for health articulates six core principles, including human autonomy, well-being, transparency, accountability, inclusiveness, and sustainability (1).

These frameworks represent an important shift from narrow technical evaluation toward a more holistic understanding of AI's societal and clinical implications. However, they remain largely aspirational, offering limited direction on how principles should be operationalized within specific workflows or organizational contexts.

Principle-based frameworks are intentionally broad to accommodate diverse healthcare systems and use cases. Yet this generality also limits their operational utility. Health systems implementing AI must make concrete decisions about workflow integration, user interfaces, escalation pathways, and governance responsibilities; areas that are only indirectly addressed by high-level ethical guidance.

Moreover, principles such as fairness or transparency may conflict in practice. For example, increasing model complexity may improve accuracy but reduce interpretability; aggressive bias mitigation may alter performance across subpopulations in unintended ways. Without explicit mechanisms for managing these trade-offs, health systems are left to navigate trustworthiness challenges on an ad hoc basis (4).

These limitations suggest the need for frameworks that bridge ethical aspirations with operational realities, explicitly addressing how AI systems function within everyday clinical and administrative work.

Traditional evaluations of AI trustworthiness focus on properties of the algorithm itself. In contrast, socio-technical theory emphasizes that system behavior arises from the interaction between technical components and social context. From this perspective, an AI model cannot be trustworthy in isolation; its trustworthiness depends on how it is used, interpreted, monitored, and governed.

This framing aligns with empirical observations that trust failures often occur despite technically sound models. For instance, clinicians may distrust accurate models if recommendations conflict with clinical intuition or workflow constraints. Conversely, poorly performing models may be trusted excessively if embedded in authoritative interfaces or institutional mandates.

Explainability has been identified as a necessary but insufficient condition for trustworthy AI, as understanding must be situated within clinical context, professional responsibility, and governance structures (5).

Understanding trustworthiness as emergent shifts analytical focus from pre-deployment validation to ongoing system performance, user behavior, and organizational accountability.

If trustworthiness is emergent, it cannot be fully established at the point of deployment. Instead, it must be continuously produced through monitoring, feedback, and adaptation. This implies that responsibility for trustworthiness extends beyond data scientists to include clinicians, operational leaders, informatics teams, and governance bodies (6).

Operationalizing trustworthy AI therefore requires explicit attention to workflow design, decision authority, failure management, and longitudinal evaluation; domains often underemphasized in technical AI research.

Trustworthy AI systems should be designed around the decisions they are intended to influence, rather than around predictive tasks alone. This requires explicit articulation of the decision context, including:

The clinical or operational decision to be supported

Decision-centered design also requires alignment with organizational incentives and constraints. For example, an AI system designed to reduce length of stay may conflict with clinician priorities related to patient safety unless decision logic and accountability are explicitly defined.

Ambiguity regarding human and AI roles is a common source of implementation failure. Trustworthy deployment requires clear delineation of tasks that are automated, tasks that are augmented, and tasks that remain exclusively human.

Role delineation should be stable and visible. Frequent, unexplained changes in AI behavior or scope can erode clinician confidence and increase cognitive burden. Importantly, delineation should include explicit override mechanisms, enabling clinicians to contest AI recommendations without penalty.

This approach mitigates automation bias while preserving the potential benefits of decision support.

Healthcare environments are characterized by uncertainty, incomplete data, and evolving conditions. Trustworthy AI systems must therefore anticipate failure rather than assume perfect performance.

Failure visibility mechanisms include confidence scores, uncertainty estimates, and alerts for out-of-distribution inputs (4). When performance degrades, systems should default to safe modes that preserve human control rather than continuing to generate misleading outputs.

Opacity during failure is particularly damaging to trust. Systems that fail silently may continue to influence decisions long after their reliability has diminished.

Operational trustworthiness depends on governance structures that define responsibility across the AI lifecycle. Governance should encompass:

Pre-deployment risk assessment and ethical review

Embedding governance into routine operations ensures that trustworthiness is actively maintained rather than retrospectively audited.

Post-deployment evaluation should extend beyond traditional performance metrics to capture system-level impact. Relevant measures include:

User adoption and override rates

To consolidate the operational principles described above, we refer to the proposed model as the Decision-Governed Trustworthy AI (DG-TAI) framework (Figure 1). The naming is intentional and descriptive rather than proprietary. It reflects the central premise that trustworthiness in healthcare AI is governed not at the level of models or predictions, but at the level of decisions specifically, how algorithmic outputs are interpreted, acted upon, overridden, and monitored within real-world clinical and operational workflows.

The DG-TAI framework does not introduce new ethical principles or technical requirements. Instead, it integrates established trustworthiness constructs into a decision-centric governance architecture that emphasizes accountability, failure visibility, and longitudinal oversight. In doing so, it provides a unifying structure that links abstract trust principles to operational mechanisms that can be implemented, evaluated, and adapted over time.

This framing is particularly relevant in healthcare settings, where AI systems rarely operate autonomously and instead shape decisions distributed across clinicians, administrators, and multidisciplinary teams. By making decision governance the organizing unit of analysis, the DG-TAI framework addresses a persistent gap between model validation and real-world system behavior.

As illustrated in Figure 1, the DG-TAI framework is organized around five interdependent layers: model outputs, workflow integration, decision execution, governance mechanisms, and observed outcomes.

At the foundational layer, AI models generate predictions or recommendations. These outputs are not treated as decisions in themselves, but as inputs to structured workflows that define how information is surfaced to users, what actions are permissible, and where discretion resides. This distinction is critical: while models produce estimates, responsibility for action remains human and institutionally governed.

The workflow layer specifies human AI role delineation, including which decisions are automated, augmented, or reserved exclusively for human judgment. Importantly, this layer also defines override pathways and escalation mechanisms, ensuring that AI systems support rather than constrain.

Decisions executed within workflows then interact with governance structures that operate continuously rather than episodically. These structures include performance monitoring, bias surveillance, incident reporting, and authority to intervene when systems deviate from expected behavior. Governance is thus embedded into routine operations rather than treated as external compliance.

Finally, outcomes such as clinical, operational, experiential, and equity-related feed back into governance processes, enabling iterative refinement. Trustworthiness is therefore conceptualized as a dynamic system property that is produced and maintained through ongoing interaction between technology, humans, and institutions.

For clinicians, the framework underscores the importance of clarity in how AI systems intersect with professional judgment. Ambiguity regarding whether AI outputs are advisory, suggestive, or prescriptive has been shown to undermine trust and contribute to both over-reliance and disengagement. Decision-centered design and explicit human-AI role delineation help mitigate these risks by aligning AI recommendations with clinicians’ mental models and accountability structures.

The emphasis on failure visibility is particularly salient in clinical contexts, where uncertainty is inherent and stakes are high. Systems that explicitly communicate confidence and limitations may initially appear less decisive, yet they are more likely to foster durable trust. Importantly, such transparency supports appropriate skepticism rather than blind reliance, reinforcing professional autonomy rather than displacing it.

Operational AI systems such as those used for staffing, scheduling, capacity management, and resource allocation present distinct trust challenges that are often underappreciated. Unlike clinical decision support tools, operational systems frequently exert indirect influence on patient care, making their effects less visible and their failures harder to detect. The framework's emphasis on embedded governance and continuous evaluation is therefore particularly relevant in operational domains.

By extending trustworthy AI principles beyond bedside decision-making, this work contributes to a more comprehensive understanding of how AI shapes healthcare delivery at scale. Operational trustworthiness requires not only technical robustness but also transparency, contestability, and alignment with organizational values, particularly with respect to equity and workforce well-being.

A recurring theme in this framework is the centrality of governance. Trustworthy AI cannot be sustained through ad hoc oversight or post hoc audits alone. Instead, it requires clearly defined ownership, escalation pathways, and authority to intervene when systems underperform or behave unexpectedly. These requirements imply a level of organizational maturity that many health systems are still developing.

From this perspective, trustworthiness is as much an institutional capability as it is a technical one. Health systems that lack data governance infrastructure, interdisciplinary oversight committees, or mechanisms for continuous monitoring may struggle to operationalize trustworthy AI, regardless of model quality. Conversely, organizations that invest in governance capacity may be better positioned to deploy AI safely and at scale.

Equity considerations cut across all dimensions of trustworthy AI. While fairness is often framed as a statistical property of models, inequities may also arise from workflow design, differential access to AI-enabled services, or uneven adoption across clinical settings. The framework's focus on system-level evaluation encourages attention to these downstream effects.

Continuous monitoring of differential outcomes across demographic and socioeconomic groups is therefore essential. Importantly, such monitoring should not be limited to clinical outcomes alone but should also encompass operational consequences, such as differential wait times, staffing burdens, or access to specialized services. By embedding equity considerations into routine evaluation, health systems can move beyond compliance toward proactive risk mitigation.

This work is conceptual in nature and does not present empirical validation of the proposed framework. While the framework is grounded in existing literature and consensus guidance, its practical utility will depend on context-specific implementation and empirical testing. Future research should examine how different elements of the framework influence trust, adoption, and outcomes across diverse healthcare settings.

Potential research directions include comparative studies of governance models, evaluations of failure visibility mechanisms, and longitudinal analyses of trust dynamics following AI deployment. Additionally, there is a need for validated metrics that capture workflow-level trustworthiness, complementing existing technical performance measures.

The World Health Organization (1) guidance establishes a foundational ethical and human-rights framework for artificial intelligence in health, emphasizing principles such as transparency, accountability, equity, and protection of human autonomy. However, it intentionally remains high-level and does not prescribe concrete operational mechanisms for implementation, monitoring, or value realization. The FUTURE-AI framework (8) advances the field by translating trustworthy AI into six core principles and 30 consensus-based best practices across the AI lifecycle, offering substantially more practical guidance for healthcare AI development and deployment. Nevertheless, FUTURE-AI focuses primarily on system-level practices and stops short of defining enterprise governance structures, decision rights, or performance management mechanisms. In contrast, the DG-TAI framework extends both WHO and FUTURE-AI by functioning as an execution-oriented governance operating model that embeds trustworthy AI principles into organizational structures, lifecycle controls, and measurable outcomes. DG-TAI explicitly links ethics and trustworthiness to accountability, continuous monitoring, and value realization, enabling health systems to operationalize trustworthy AI at scale rather than treating it solely as a design-time or compliance consideration.

While numerous frameworks have articulated principles for trustworthy or responsible AI, the DG-TAI framework contributes to the literature in three distinct ways.

First, it repositions trustworthiness as a runtime system property rather than a design-time attribute. Existing frameworks largely emphasize pre-deployment validation and ethical compliance. In contrast, DG-TAI foregrounds post-deployment behavior, recognizing that trust is shaped by how systems perform under real-world conditions, including uncertainty, drift, and organizational change.

Second, the framework elevates decisions as the primary unit of governance, rather than models or predictions. This shift has practical and ethical significance. Decisions are observable, auditable, and attributable, making them more amenable to governance than abstract model properties. By anchoring trustworthiness in decision execution, the framework aligns AI oversight with established clinical and administrative accountability structures.

Third, DG-TAI explicitly extends trustworthy AI considerations to operational workflows, including staffing, scheduling, capacity management, and resource allocation. While prior work has focused predominantly on clinical decision support, operational AI systems often exert broader systemic influence and may generate inequities that are less visible but equally consequential. The framework treats clinical and operational AI symmetrically, applying the same trust, governance, and evaluation principles across both domains.

Importantly, the contribution of this work lies not in introducing new ethical norms, but in integrating existing principles into a coherent operational architecture. This synthesis enables health systems to move from aspirational trustworthiness to implementable, monitorable practice.

The DG-TAI framework suggests several directions for future empirical research. First, it motivates the development of metrics that assess trustworthiness at the decision and workflow levels, complementing traditional model-centric performance measures. Second, it invites comparative evaluation of governance models, examining how different ownership and escalation structures influence trust, adoption, and outcomes. Third, it highlights the need for longitudinal studies that examine how trust evolves over time as AI systems and organizational contexts change.

By offering a decision-governed lens, the framework provides a foundation for testable hypotheses without presupposing specific technologies or use cases. As such, it is designed to remain applicable across clinical domains, operational functions, and health system contexts.

Consider a large, multi-hospital healthcare system seeks to deploy an AI-enabled workforce forecasting and staffing optimization solution to address persistent nurse shortages, reduce reliance on premium labor, and improve patient safety. Applying the DG-TAI framework, the organization begins by establishing a formal governance structure that defines decision rights and accountability across HR, clinical operations, IT, data governance, and compliance. Furthermore, Executive sponsors, model owners, data stewards, and a trust or ethics review function are identified to oversee workforce equity, clinician well-being, and regulatory considerations.

Rather than proceeding directly to enterprise-wide rollout, DG-TAI requires deployment as a controlled pilot within a limited set of units or facilities. During the design and pre-deployment phase, historical staffing, census, acuity, and scheduling data supporting the pilot are subjected to rigorous data quality, representativeness, and bias assessments. The model is evaluated for differential impact across shifts, units, and demographic proxies, and technical risks such as robustness and generalizability are assessed in the context of the pilot environment. Advancement beyond the pilot is contingent on meeting predefined trust and performance thresholds, not solely on predictive accuracy.

During the pilot phase, DG-TAI operationalizes continuous monitoring through a defined set of key performance indicators spanning technical, ethical, and operational dimensions. These include forecast accuracy and drift, override and exception rates, staffing equity indicators, overtime hours, vacancy coverage, and clinician experience proxies. Governance forums review these metrics at regular intervals, enabling rapid identification of unintended consequences and informed decision-making about model refinement.

Only after the pilot demonstrates sustained performance, acceptable equity impacts, and operational value does DG-TAI authorize scaled deployment across additional units or the broader enterprise. Post-scale, monitoring and governance mechanisms remain active, with formal escalation and remediation pathways triggered by threshold breaches or contextual changes. Insights from the pilot and early scale phases are systematically captured and fed back into the organization's AI maturity model, strengthening future implementations. This pilot-to-scale approach illustrates how DG-TAI operationalizes trustworthy AI as a phased, accountable process, bridging ethical intent, real-world execution, and sustained value realization in complex healthcare settings.