Patients and the public were not involved in the design or conduct of this systematic review.

Articles on risk prediction models for postoperative delirium in older adult patients with hip fractures were searched until March 15, 2023, using the following databases: the Cochrane Library, PubMed, Web of Science, and Ovid. The following terms are used: “hip fracture” and “delirium.” Our complete search string for PubMed was “(hip fracture OR trochanteric fracture OR subtrochanteric fracture OR hip joint implantation OR hip replacement OR hip arthroplasty) AND (delirium OR disturbance of consciousness OR cognitive impairment OR excitement OR excitement OR POD OR POCD).” The search is limited to Titles/Abstract and the references of all original articles were screened (See Appendix 1). The language of the articles was English.

Articles meeting the following criteria were included: (1) Study designs, cohort study or case–control study; (2) Populations, older adult hip fracture patients with an age over 60 years; (3) Outcome, postoperative delirium; and (4) the research content, tools, and methods used for the construction of the risk prediction model were given in detail, and internal or external validation was carried out after the establishment of the prediction model. We excluded articles where (1) the development process or method for establishing the model was not described; (2) the model’s predictors cannot be widely evaluated or accurately measured in clinical practice; (3) full-text of the article was not available; and (4) Repeated publications.

Two researchers independently screened the literature, extracted the data, and cross-checked the data. In the case of disagreement, they consulted a third party. For literature screening, we first read the title and abstract, and after excluding irrelevant literature, we further read the full text to determine inclusion. The extracted data included the first author, time of publication, country, research type, participants, modeling sample size and outcome events, modeling methods and verification model method, criteria for POD (Postoperative Delirium), model performance including Area Under Curve (ACU) and calibration methods, number and names of predictive factors, and risk factor assignment/risk stratification method.

The meta-analysis used Stata (version 13.0) to extract research data and generate the forest map. In our meta-analysis, the Odds Ratio (OR) and corresponding 95% Confidence Interval (CI) were combined to explore the relationship between the risk factors and POD in older adult patients with hip fracture. We detected heterogeneity using the Q test. When p < 0.1 or I² > 50%, the random effect model is selected; When p > 0.1 and I² < 50%, select the fixed effect model. After a combined analysis, it was considered statistically significant when p < 0.05. A sensitivity analysis was conducted to detect sources of heterogeneity by removing each study from the meta-analysis independently. Potential publication bias was judged by Begg’s test and Egger’s test; p < 0.05 was considered significant. If there was a potential bias, the trim-and-fill method was used to reassess.

Quality was assessed using the Newcastle-Ottawa scale (NOS), which includes 3 major dimensions: selection, comparability, and exposure. The evaluation was scored out of 9, with a score of ≥7 being good-quality literature and < 7 being inferior-quality literature.

The two researchers (HYQ and YY) independently assessed the risk of bias in the selected studies using PROBAST, and a third party (TP) determined the difference. The PROBAST, which was developed by Wolff and his team in 2019, includes a risk of bias assessment and an applicability evaluation (8). PROBAST is organized into four domains, including participants, predictors, outcomes, and analysis. Based on the evaluation results of each domain, the risk of bias and applicability of the prediction model were obtained (9).

Predictive performance is mainly evaluated from the perspectives of discrimination and calibration. The discrimination is measured by the AUC (AUC ≥ 0.7 indicates good model discrimination), among which we believe that the AUC of external testing is more representative than the AUC of internal testing. The calibration is evaluated through the Hosmer-Lemeshow test (when the Hosmer-Lemeshow test p > 0.05, it indicates good model fit; otherwise, it is considered poor model fit) and the calibration plots (when the calibration slope is close to 1, it is considered that the model fits well).

Initially, the researchers identified 2,409 studies. After screening, the final analysis included 8 studies (7, 10–16). In the evaluation of literature quality, 8 (≥7 points) were of high-quality. The details were provided in Figure 1.

A total of eight risk prediction models for postoperative delirium in older adult hip fracture patients were included, including three studies conducted in the United States of America (USA), four in China, and one in Australia and New Zealand (7, 10–16). In terms of research type, one was a prospective cohort study (10), one was a case–control study (16), and the other six were retrospective cohort studies (10). The earliest risk prediction model was published in 2006 (10), and six articles were published in the last 3 years (7, 11–14, 16). Five studies (10, 12, 13, 15, 16) defined the participants as patients with hip fracture or proximal femur fracture aged 65 years or older, and the other three studies (7, 11, 14) defined age as 60 years or older, resulting in a high level of homogeneity (Table 1).

In the included models, the modeling sample size was 77 ~ 22,563, and the incidence of delirium was 13.04% ~ 48.05%. In terms of modeling methods, three studies used single factor analysis to select the factors related to postoperative delirium in older adult hip fracture patients, and then used logistic regression to select independent predictive factors and modeling (10, 12, 15); one study used Lasso regression and logistic regression modeling (14); there are two studies using a recursive random forest (RF) algorithm to identify variables that may be relevant; finally, the Machine learning (ML) algorithm constructs the model (7, 16); and two studies used stepwise regression analysis to obtain the prediction model (11, 13). As for the method of validating the model, one study used internal validation and external validation (14), while five studies only used internal validation (7, 11, 12, 15, 16) (Table 2).

Of the eight included studies, at most 9 predictors were included (11), and at least 3 predictors were included (14). In the present systematic review, the most common predictors of postoperative delirium in older adult hip fracture patients were preoperative dementia or history of dementia (n = 5), ASA classification (n = 4), and age (n = 3). In terms of the risk factor assignment, three studies assigned the value of prediction factors by OR values of logistic regression, and the scores were the sum of the scores of each prediction factor for final risk judgments (11, 13, 15). Based on β coefficient of logistic regression, three other studies assigned weight to each predictor (10, 12, 14). The last two studies generated specific delirium prediction models based on machine learning to determine the weights of prediction factors, and then predicted the probability of delirium occurrence (7, 16), as detailed in Table 3.

We performed a meta-analysis for preoperative dementia or history of dementia, ASA classification, and age. Due to the inability to extract the required data from literature such as Oosterhoff JHF (7), a meta-analysis was conducted on the remaining studies after exclusion. The results indicated that preoperative dementia or history of dementia, ASA classification, and age were independent risk factors for postoperative delirium in older adult patients with hip fracture. The results are presented in Table 4. As an example, a sensitivity analysis was drawn for dementia. We further explored the source of heterogeneity by removing each study from the meta-analysis independently. The results showed that ignoring any of the enrolled studies did not significantly change the effect of the dementia on the combined meta-analysis for POD. That indicated that the overall results were stable and reliable (Figure 2). In the meta-analysis for dementia, Begg’s test (p = 0.734) and Egger’s test (p = 0.716) determined no significant publication bias (Figures 3A,B).

In the eight included articles, five studies were at high risk of bias in the bias risk assessment (10, 12, 13, 15, 16). The high-risk areas were mainly participants and statistical analysis. Two studies (11, 14) were at unclear risk, and the remaining one (7) was at low bias risk. In terms of applicability evaluation, six studies were low risk of applicability (7, 10–14), and two studies (15, 16) were unclear risk of applicability levels (Table 5).

We evaluated the performance of the model from the perspectives of discrimination and calibration. In terms of discrimination, Zhang et al. (15) reported that the AUC was only 0.67, which indicates that the model has poor discrimination; the modeling AUC in both articles (7, 16) is greater than 0.7; Kim et al. and Oberai et al. (11, 12) reported that both the modeling and internal validation AUC were greater than 0.7; and the model developed by Yang et al. (14) performs best in discrimination, with AUC > 0.7 for both modeling and external validation. In terms of calibration, three articles (12, 14, 15) used the Hosmer-Lemeshow test, but the model developed by Oberai et al. (12) exhibited a p < 0.05 test result, indicating poor model fitting; three articles (7, 11, 15) used calibration plots, with calibration slopes close to 1 and indicated good calibration. The model of Zhang et al. (15) showed good calibration on both methods.

These eight prediction models in the collected studies include many prediction factors, such as socio-demographic information, medical information, scale test results, and clinical information, which can be obtained through simple and rapid inquiry or evaluation. Although the number and type of prediction factors in each model differed, there are some commonalities. Among them, a history of preoperative dementia or dementia history, ASA classification, and age were high correlated with postoperative delirium in older adult hip fracture patients, and meta-analysis suggests that they are independent risk factors, which is highly consistent with many other studies exploring the risk factors of postoperative delirium in older adult hip fracture patients (17–20). A history of preoperative dementia or dementia is a predictive factor of concern, and preoperative dementia patients are a special subpopulation (17). As a cognitive dysfunction, although there is no international consensus on the effect of preoperative dementia or dementia history on POD, it has been proven to be correlated with postoperative delirium (21–23). Rong et al. conducted a meta-analysis including 22 articles on the risk factors of postoperative delirium after knee and/or hip replacement, of which 16 articles were on older adult patients with hip replacement (21). They found that dementia is a risk factor for postoperative delirium (21). Lee et al. conducted a prospective cohort study on older adult hip fracture patients and found that the incidence of postoperative delirium in patients with preoperative dementia or dementia history was 2.1 times higher than that in the control group (23). A history of preoperative dementia or dementia may cause brain metabolic disorders and polyamine pathway disorders, which may contribute to postoperative delirium (24). Change in polyamine level caused by the imbalance in the polyamine pathway will result in abnormal ion channel and ion glutamate receptors, followed by electrolyte disorder. At the same time, electrolyte disorder can lead to microcirculation disorder, which plays a particularly important role in the occurrence of postoperative delirium (25–27).

The ASA classification is used to assess the general disease status and overall health status of patients and is one of the most valuable methods for preoperative determination of surgical and anesthetic risk (28). Although the ASA classification was originally designed as an anesthetic risk assessment system, it is now widely used to predict perioperative risk and mortality (29, 30). Hackett et al. also believed that the higher the ASA classification, the worse the overall health of patients, and the more significantly increased postoperative complications (31). In addition, ASA classification can be used as a risk factor for postoperative death (28), and also as an independent risk factor for postoperative delirium in older adult hip fracture patients (32). Therefore, for patients with high ASA classification, medical personnel should strengthen preoperative continuous monitoring, postoperative prevention and treatment, improve the compensatory ability of each organ, and effectively prevent postoperative delirium.

Age is recognized as an independent risk factor for postoperative delirium (33, 34). Studies have confirmed a correlation between age and postoperative delirium in older adult hip fracture patients (6, 17). Haynes et al. studied 18,754 older adult hip fracture patients and confirmed that age was an independent predictor of postoperative delirium (17). The reason may be that with the increasing age, degenerative changes in the brain parenchyma of older adult patients occur, such as aging of nerve cells, reduction of cerebral blood flow perfusion, and changes in the content of central neurotransmitters, among which the change in the central neurotransmitters content is an influential cause of delirium (35, 36). Due to the weakened function of important organs such as the heart, brain, and lungs, the compensatory ability of older adult patients is significantly reduced, leading to reduced tolerance to anesthesia and surgery. This can result in severe hemodynamic fluctuations, stimulating the body to release inflammatory factors. These inflammatory factors can induce inflammatory responses in the central nervous system, causing changes in the cognitive level of patients and even postoperative delirium (37–39).

The risk of bias in prediction models is closely related to the source of participants, definition and evaluation of prediction factors, classification and definition of outcomes, and statistical analysis. The present systematic review included eight articles, of which five studies had a high risk of bias (10, 12, 13, 15, 16), two studies had uncertain bias risk (11, 14), and one study had a low risk of bias (7). The main reasons behind this are: (1) risk of bias in data sources; (2) insufficient sample size; (3) unreasonable processing of independent variables; (4) defects in processing methods for missing data; (5) adoption of single factor analysis to screen prediction factors; (6) lack of performance evaluation of prediction models; and (7) failure to consider whether there are problems with model fitting. PROBAST points out that data from randomized controlled trials, registered data, prospective cohort studies, Nested case–control studies, or case-cohort studies are superior to retrospective cohort studies and traditional case–control studies (8). However, only one in the 8 selected studies comes from prospective cohort study (10). In terms of sample size, PROBAST requires that model development studies should have more than 20 events per variable (EPV) to avoid overfitting of the model; model validation studies should include at least 100 subjects with outcomes (40). Most studies fail to meet the requires in the sample size of modeling or model verification, which increases the risk that the prediction model may contain incorrect predictors or fails to include significant predictors (7, 10, 13–16). Regarding the processing methods of independent variables, two studies simply classified continuous variables into binary variables (10, 13), and one study transformed continuous variables into ≥2 category variables, leading to losing lots of useful information and even reducing the predictive power of the model (11). For the processing of missing data, two studies had no missing data (10, 15), one study used multiple imputation to deal with missing values (7), while the remaining studies directly excluded the inclusion of missing data and used complete data analysis (11–14, 16). The use of univariate analysis to screen predictors is a routine strategy in model development studies. Three studies used univariate analysis to select relevant factors, but researchers do not recommend it as a basis for screening predictive factors (10, 12, 15). In univariate analyses, models end up incorporating inappropriate predictors or rejecting valid predictors because of collinearity between independent variables (41). Thus, according to the guidelines of the Transparent Reporting of a Multivariate Prediction Model for Individual Prognosis or Diagnosis (TRIPOD), it is recommended to use the stepwise regression method or appropriately adjust the significance level during univariate analysis (42). In terms of model performance, only five studies reported both AUC and calibration, and used Hosmer-Lemeshow tests or calibration plots to describe the calibration (7, 11, 12, 14, 15). Among them, the p value obtained by the Hosmer-Lemeshow test cannot be used to quantify the model calibration (43). It is recommended to use or combine calibration plots to describe the calibration of the prediction model. Three studies used calibration plots (7, 11, 15), and one study used both methods (15). Model performance indicator tend to have optimistic biases due to overfitting or the selection of better thresholds. Therefore, internal verification through Self-service Sampling or cross validation is necessary. Six of the included studies conducted internal testing (7, 11, 12, 14–16), three of which used the randomized splitting method, an inefficiency testing method (11, 12, 14); two of which used a combination of randomized splitting and K-fold (7, 16); and the remining one adopted the Self-service Sampling (15). One study used both internal validation and external validation, and the AUC value for external validation was 0.84 (14). In terms of model applicability, only two studies were unclear about the risk (15, 16), and the other studies had good applicability (7, 10–14). The overall applicability of the eight studies was good.