A deterministic large language model (LLM) framework for safe, protocol-adherent clinical decision support: application in hemodialysis anemia management (AnemiaCare HDs) - Arriola-Montenegro, Jose; Thongprayoon, Charat; Bizer, Benjamin; Miao, Jing; Ordaya-Gonzales, Karina; Craici, Iasmina M.; Cheungpasitporn, Wisit (Frontiers in Artificial Intelligence, 2025)
BackgroundLarge language models (LLMs) show promise for clinical decision support but often deviate from evidence-based protocols, raising safety and regulatory concerns. Anemia management in hemodialysis patients requires strict adherence to erythropoiesis-stimulating agent (ESA) and intravenous (IV) iron dosing rules, making it a high-risk use case for uncontrolled model behavior. To address this gap, we developed AnemiaCare HD, a deterministic LLM framework engineered to deliver transparent, reproducible, and protocol-adherent clinical recommendations.MethodsAnemiaCare HD was evaluated using 600 simulated hemodialysis anemia scenarios derived from a standardized institutional protocol. The model required six fixed clinical inputs (hemoglobin, hemoglobin rate of change, trend direction, transferrin saturation, ferritin, and current ESA dose). Phase 1 tested a loosely structured prompt. Phase 2 implemented deterministic prompt logic incorporating ESA kinetics, iron dosing rules, mandatory timing safeguards, and embedded safety alerts. Two independent nephrologists assessed protocol adherence.ResultsIn Phase 1, only 96 of 300 cases (32%) aligned with protocol recommendations, with common errors in ESA titration, iron dosing, and timing violations. In Phase 1, loosely structured prompting produced variable outputs, with only 96 of 300 simulated cases (32%) fully protocol-adherent and frequent unsafe recommendations. In contrast, deterministic prompting in Phase 2 resulted in 100% adherence across all 300 cases, eliminating protocol deviations, unsafe iron dosing, and timing violations (p < 0.001). In Phase 2, deterministic encoding achieved full protocol adherence (300/300, 100%), eliminating unsafe or premature recommendations (p < 0.001 vs. Phase 1) and consistently generating structured, rationale-based outputs.ConclusionDeterministic LLM engineering enables safe, fully protocol-compliant clinical decision support in high-risk therapeutic domains. AnemiaCare HD demonstrates the viability of regulatory-aligned, auditable LLM frameworks for clinical use, although real-world integration and prospective validation remain necessary next steps.