The use of artificial intelligence (AI) is becoming increasingly important for medical use. Regarding prognostic abilities, there are already publications suggesting that AI-based image morphological recognition of stroke extent has potential comparable to that of an experienced neuroanatomist (1). In the significantly more complex detection of acute common and severe diseases based on clinical data, Levine et al. (1) demonstrated that Chat GPT 3 outperformed non-medically trained individuals, but not physicians. Bentley et al. (2) used machine-learning-based image recognition software to predict hemorrhagic transformation after intravenous thrombolysis in ischemic stroke. Using supervised machine learning algorithms, another research group was able to predict the outcome of patients with ischemic stroke after intra-arterial therapy with an accuracy of ~70% (3). Despite these promising results, GPT has not yet been shown to predict the 6-month outcome after traumatic brain injury due to insufficient specificity (4). However, promising results have been reported regarding GPT's potential for outcome prediction in aneurysmal subarachnoid hemorrhage (5).

MCA infarction is a severe condition with high mortality and a major impact on the patient's quality of life in cases of survival. In malignant MCA Infarction, decompressive hemicraniectomy is the ultima ratio for preserving the patient's life (6). To date, the prediction of the prognosis for these patients remains extremely difficult.

The functionality of modern AI is based on complex digital neural networks, which are created based on real data and are capable of processing complex tasks using deep learning techniques (7). One of the most advanced AI-based applications currently available for public use is GPT. GPT is a language model that was developed and trained by the company OpenAI to generate answers that are as human-like as possible (8). GPT processes data and relates them to each other within a network and creates a so-called “transformer architecture” to enable precise categorization within the respective context (8, 9).

The present study is the first to investigate GPT's current ability to process complex real-life patient data into prognostic estimation of the patient's clinical outcome.

This retrospective analysis investigated the capability of deriving a prognosis using a single data modality input from patient data. Data were collected from patients admitted to the hospital with malignant MCA infarction who underwent emergency decompressive hemicraniectomy. To further enhance prognostic assessment, the study investigated whether providing context for decision-making can contribute to improving predictive accuracy. Data from 100 patients who underwent decompressive hemicraniectomy for MCA infarction at Leipzig University Hospital between 2016 and 2023 were assessed. To provide the large language model (LLM) with comprehensive input, patient-specific parameters (age, gender, previous cardiac diseases, intake of blood-thinning medication, laboratory parameters such as leukocytes, platelets, CRP, and preoperative pTT), disease-specific parameters [infarct size, hemorrhagic transformation, pupil status, mRS, and Glasgow Coma Scale (GCS)], and therapy-specific parameters (volume and diameter of the decompression and hemoglobin levels before and after the procedure) were sampled and provided to the AI anonymously. These parameters have largely already been associated with the prognosis after decompressive hemicraniectomy in previous studies (10–15).

The infarct volume was calculated using the Brainlab Suite's volumetric function (Brainlab, Feldkirchen, Germany) (16). To depict the extent of the decompressive hemicraniectomy, both the AP diameter usually given in the literature and the surface area of the decompressed area were specified according to the formula As = π[(d/2)² + h²] (17).

The neurological outcome of the patients was assessed using the modified Rankin Scale (mRS) (18). In accordance with the existing prospective randomized studies—DECIMAL (19), HAMLET (20), DESTINY (21), and DESTINY II (22)—the mRS was also included in the analysis at the time of discharge, and after 3 months, 6 months, and 1 year.

The data were provided to ChatGPT in a standardized chat prompt. For our investigation, we utilized two versions of GPT: GPT-4, released in March 2023, and the advanced version, GPT-4 Omni, released in May 2024 (23). A total of five questions were formulated, each of which had to be answered with a yes/no response. Each chat prompt was offered to both versions of GPT with and without context-enrichment (CE) to provide the LLM a defined base for reasoning. As CE, we chose the current 2019 ASA/AHA guideline (6) as well as a meta-analysis of the prospective randomized studies (24) in patients under 60 years of age, and the prospective randomized study DESTINY II (22) in patients over 60 years of age. Each question was asked a total of 3 times to consider divergent answers. The mean of the given answers was documented. The answers were scored as follows: three times “no” (score: 0), two times “no” (score: 0.33), one time “no” (score: 0.66), and three times “yes” (score: 1.00). GPT was asked to evaluate the survival during the in-hospital stay, as well as the functional outcome at discharge, 3, 6, and 12 months in a yes or no answer. Favorable (mRS 0–4) and non-favorable outcomes (mRS 5–6) were dichotomized as defined in the prospective randomized studies (24). An exemplary chat prompt is provided in Supplementary Figures 1, 2.

Data were entered into an anonymized database, and this database was analyzed with SPSS (IBM Corp., Released 2023. IBM SPSS Statistics for Windows, Version 29.0.2.0, Armonk, NY: IBM Corp). First, we performed a the descriptive analysis of our cohort (Table 1). GPT's answers were subjected to a receiver operating characteristic analysis (ROC) to determine the area under the curve (AUC), sensitivity, and specificity stated with the 95% confidence interval (CI) (Figure 1). The Youden index was calculated to define the optimal cutoff in the case of divergent answers (Table 2). After dichotomization according to the determined cutoff, GPT's answers were tested for significance using a chi-squared test.

In an additional prompt, the pre-stroke mRS was included to refine the mode (Supplementary Figure 4). GPT-4 could not be included in the following analysis as it had been replaced by OpenAI with a more recent version. GPT-4 Omni was asked to predict the mRS at the time of discharge. A delta (Δ) between the pre-stroke mRS and the mRS at the time of discharge was calculated for GPT's estimation, as well as the real mRS (Figure 2). Subsequently, the ΔmRS was assessed by Pearson's correlation coefficient.

Our study shows that GPT can estimate prognosis for patients with malignant infarcts of the middle cerebral artery who have undergone decompressive hemicraniectomy based on patient profiles.

After dividing the patients into cohorts of over and under 60 years of age, analogous to the inclusion criteria of the randomized studies [HAMLET (20), DESTINY/II (21, 22), and DECIMAL (19)], our study group included a higher percentage of male patients. Additionally, patients in our cohort were older at the time of the event. The 1-year mortality rate exceeded the mortality rate stated in the randomized studies, especially in the group of patients over 60 years of age. The differences in mean patient age, gender, and long-term survival are likely due to a complex combination of factors in the population groups, healthcare systems, and possibly due to selection bias in the respective study designs.

Despite the differences in the patient cohort, the CE versions GPT4.0 and GPT-4 Omni are able to predict the patient's survival with robust accuracy. Interestingly, the earlier version GPT-4 reaches a higher AUC than GPT-4 Omni. GPT-4 Omni, in turn, achieves the highest statistical significance in the chi-squared analysis after Youden index-based dichotomization of multiple answers. Non-CE GPT versions only reach insufficient AUCs. The present results suggest that CE GPTs may be more capable of estimating survival outcomes.

In the subgroup analysis, significance was achieved for ≥61-year-old patients and <61-year-old patients by GPT-4 Omni (CE), but only for ≥61-year-olds in the model GPT-4. The significance in GPT-4 Omni (CE) is consistent with the main analysis and indicates that the results are valid regardless of the investigated age groups, underlining the advantage of CE. However, the implications of the results from the more recently developed GPT-4 remain unclear, though they may reflect progress in source-based reasoning abilities seen in GPT-4 Omni.

In addition to the question of survival, the LLM was also asked to predict functional outcomes as measured by the mRS. GPT was unable to provide sufficient answers, regardless of the version used. In a further series of tests, GPT4 Omni (CE) was provided with the pre-stroke mRS as a baseline functional status for each individual patient. Here, the functional outcome could also be predicted with a significant correlation by the GPT. The results suggest that a baseline might be vital for the LLM's reasoning process when making predictive estimations of manageable complexity. This additional input supports the theory that initial insufficient answers may be seen as the expression of hallucinations. This “data hallucination” can occur when an AI is working on a topic on which it has not been explicitly trained. As a result, fictitious answers may be generated without a founded basis for reasoning (25, 26). Another aspect is that GPT seems not to be able to adequately include the concept and perception of time into the calculations, adding another layer of complexity to the question of time-dependent functional recovery (27).

To understand the limitations of AI, its basic functioning must first be understood. In Order to calculate the propability of the next correct word, each word is related to the previous one and to each other. This highly complex calculation approach is beyond human control and monitoring, making it impossible to understand the rationale behind a calculation. This bears the danger of arbitrary surrogate parameters being used for calculation (25).

AI is fundamentally limited by the data on which it is trained. An existing bias (ethnic group, patient selection, infrastructural characteristics, etc.) is continued by the AI and can produce a result that does not correspond to existing reality. The differences in the patient population in the existing studies and our collective alone, therefore, inevitably lead to inaccuracies. It is all the more remarkable that, despite these differences, a robust association with CE-GPT's prediction of short-term outcome was achieved.

Another aspect that must always be considered when using AI-based systems is that of ethics. When we weigh up a prognostic decision as treating doctors, we include hard and soft data and factors in our decision-making. Similarly, especially soft factors such as contextual or environmental conditions will not be represented in AI evaluations. The extent to which AI is involved in this decision-making process is a very delicate question and must always remain the subject of controversial debate.

Additionally, access to AI as a source of medical information is not limited to medically trained professionals. Non-medical users have equal access to the AI tool through interfaces such as chatbots. Unlike medical professionals, however, they lack the ability to critically assess, contextualize, and interpret the AI's response. Going forward, great emphasis should be placed on guiding non-medically trained persons to prevent harm by misinterpretation or false conclusions. There are approaches to implement AI-based, machine learning driven prediction models (28, 29). However, such models are less prone to hallucinations due to their targeted use of validated parameters, yet have not been integrated into LLMs.

At the present time, the AI-based language model GPT, in versions GPT-4 and GPT-4 Omni, is able to predict the short-term outcome of patients with decompressive hemicraniectomy after malignant MCA infarction with a significant degree of certainty based on freely available data. However, the question of time-dependent functional outcome appears more complex and does not yield any meaningful results, with a high risk of producing data hallucinations. Future studies should focus on two specific objectives: first, identifying ways to further improve GPT's prognostic abilities; second, understanding AI decision paths to decipher the black box of decision-making before implementing AI-based decision-making in practical healthcare.