A comprehensive search was conducted in five databases: PubMed, Embase, Web of Science, the Cochrane Library, and CNKI, with a search to January 1, 2025. The search strategy for PubMed was as follows:

Appendix 1 contains detailed search strategies for 5 databases.

We included studies on developing and validating predictive models, regardless of whether external validation was performed.

The inclusion criteria were based on the key elements of the PICOT system. The PICOTS statement was as follows:

We excluded studies from which model performance metrics could not be extracted, studies without full text available, and studies published as letters, protocols, reviews, or case reports. Study selection and data extraction.

All search results were imported into Zotero for merging and deduplication. Two evidence-based trained reviewers (ZZH and LJ) screened the titles and abstracts, obtaining the full text of potentially eligible studies. The two reviewers independently assessed whether the studies met the inclusion criteria, with any disagreements resolved through discussion. Discrepancies were resolved by a third reviewer when necessary.

We used a standardized Excel template developed by Borja M. Fernandez Felix (17) to extract data and assess the risk of bias and applicability of the predictive model studies. This template combines the CHARMS (17) and PROBAST (18) tools into one file, streamlining and standardizing data extraction and bias risk assessment, reducing error risk, and improving reviewers’ consistency. The CHARMS worksheet follows Moons et al.’s framework (18, 19). The data extraction table was built around 11 CHARMS domains. The extracted data included study characteristics (author, publication year, title, model name), study design (data source, recruitment methods, study setting, number of centers, inclusion/exclusion criteria, participant characteristics such as age and gender), outcomes, predictors, data characteristics (sample size, events, EPV, missing data handling), model development, performance, validation, and final model details.

Reviewers filled in all highlighted cells and used dropdown menus or free text responses. If information was unavailable, reviewers indicated “NA.” The PROBAST tool assessed bias and applicability across four domains: participants, predictors, outcomes, and analysis. Bias risk was rated as low, high, or unclear.

Studies were compared based on their characteristics. Research characteristics, quality, and model performance data were tabulated. Descriptive statistics summarize study types, outcomes, statistical methods, and predictors. A meta-analysis of externally validated predictive models was performed using Stata 18.0, with the primary effect size being the area under the receiver operating characteristic curve (AUC). The standard error (SE) for each study’s AUC was calculated based on sample size and AUC using the formula (20) SE=AUC×(1−AUC)/n, The data were declared using the meta set command, with the effect size being AUC and the standard error pre-calculated. A random-effects model was used for the meta-analysis, and heterogeneity was assessed using I².

Table 1 summarizes the characteristics of the included studies, including author, year, study design, enrolment period, study setting, study region, and participant demographics. Among the 23 included studies, 6 (9, 33, 35–38) focused exclusively on model development, 7 (21–23, 28, 29, 31, 32)on model development and internal validation, 7 (24–27, 30, 34, 39)on model development with both internal and external validation, and 2 (41, 42) on external validation alone. Notably, one study (41) that focused solely on external validation also recalibrated the original model (35) and modified its predictive factors.

Among the 21 studies focused on model development, the diagnostic criteria for PEP varied. The Cotton criteria (43) were employed by eight studies (38.1%), while the Atlanta criteria (44)and the revised Atlanta criteria (45) were used by six studies (28.6%) and four studies (19%), respectively., and the Guidelines (46) were applied in one study (4.8%). Two studies (9.5%) did not follow any standardized diagnostic criteria, instead using descriptive methods. Regarding the timing of PEP onset, most studies focused on the early postoperative period. A range of <24 hours was reported in eight studies (38.1%), followed by <72 hours and <48 hours in two studies (9.5%) each. One study (4.8%) reported PEP onset within the 24–48 hours and <5-day windows. Long-term observations were less common, with one study (4.8%) observing PEP onset for <2 weeks, another (4.8%) for 24 hours to 2 weeks, and one study (4.8%) for <30 days.

Twenty-one prediction models for PEP were developed across 23 studies. Of these, nine models (9/21) included external validation sets. Table 2 presents the prediction models, development methods, and performance metrics. Among the 21 models, 16 (16/28) used traditional Logistic Regression (LR) alone, while five models (5/21) incorporated machine learning algorithms. These machine learning methods included Gradient Boosting (GB), Random Forest (RF), Gradient-Boosted Machines (GBM), Deep Learning (DL), and Classification and Regression Tree (CART).

All models reported AUC values, ranging from 0.62 to 0.98 across the 21 studies included. While calibration is crucial for clinical applications, ensuring that a model’s predicted probabilities align with the actual occurrence of events, it enhances the model’s credibility and practical utility (47). Different studies have employed various calibration evaluation techniques. Among the studies included, calibration plots were used in (15, 21–24, 26, 27, 29, 32, 34, 39), as these plots provide a clear visualization of the relationship between predicted probabilities and actual outcomes. The Hosmer-Lemeshow test was utilized in (5, 23, 31, 32, 35, 39), a commonly used statistical method that assesses model fit by comparing the observed event frequencies across different predicted probability groups (47). More advanced calibration metrics, such as the Brier score, calibration slope, and CITL (48), were reported in (21), offering more detailed calibration measurements. The Brier score, in particular, is a key metric for evaluating the accuracy of predictions, considering the differences between predicted probabilities and actual outcomes. Six studies (28, 30, 33, 37, 38, 40) did not report any calibration methods, which hinders a comprehensive assessment of these models’ practical applicability. The lack of calibration evaluation in these studies raises concerns about potential biases in their clinical application.

The most common method for selecting candidate predictors was univariable analysis, used in 13 studies (61.9%). Four studies (19.0%) relied on prior knowledge. Two studies combined feature importance from the Gradient Boosting (GB) model with univariable significance (p < 0.2) and machine learning techniques. One study used exploratory factor analysis (EFA) for factor selection.

For final predictor selection, stepwise selection was the most common, applied in 8 studies (38.1%). Backward elimination was used in 6 studies (28.6%), and LASSO selection in 2 studies (9.5%). The GBM algorithm (with all predictors) was used in 2 studies (9.5%). Forward selection and recursive partitioning appeared in 1 study (4.8%) each, and a pre-specified model in 1 study (4.8%).

These factors can be categorized based on their modifiability and clinical relevance: modifiable factors include difficult cannulation (13 models), operator experience (4 models), history of bile duct stones (4 model), and precut sphincterotomy (4 models), which can be addressed during the procedure to guide preventive strategies.

Disease-related factors, such as a history of pancreatitis (6 models), pancreatic duct injection/visualization (7 models), and pancreatic sphincterotomy (7 models), are important for prediction but cannot be altered during surgery.

Demographic factors, such as female sex (9 models) and age (7 models), are based on patient characteristics and are typically known before the procedure.

Less common predictors include hypertension (3 models), neutrophil count (2 models), hypoalbuminemia (2 models), and direct bilirubin (2 models), which play a role in prediction but appear less frequently in the models (Figure 2).

The bias risk and applicability assessments of the 21 model development studies are summarized in Table 3. Three studies (25, 26, 39) (14.3%) were rated as having low overall bias risk, while 10 studies (15, 22, 24, 27, 30, 32–36) (47.6%) had an unclear bias risk, and 8 studies (21, 23, 28, 29, 31, 37, 38, 40) (38.1%) had a high risk of bias (Figure 3). Based on the PROBAST tool assessment, the sources of bias primarily included unclear definitions of predictor variables, inadequate sample sizes and EPV (events per variable) that did not meet recommended values, insufficient explanation of missing data handling, absence of external validation sets, and inadequate model calibration and performance interpretation.

Regarding applicability, 7 studies (15, 24, 26, 27, 31, 34, 35) (33.3%) demonstrated good performance in terms of the representativeness of the study population, clinical relevance of predictive factors, and the application of outcome measures. For 10 studies (21, 23, 25, 29, 30, 33, 36–39)(47.6%), the applicability was unclear, while 4 studies (23, 28, 32, 40) (19.0%) raised significant concerns regarding applicability. The EPV range for these four studies was between 2.2 and 8.3, below the recommended minimum of 10. These studies had sample sizes ranging from 269 to 559 participants, which were insufficient for stable model development and validation. None of these studies performed external validation, nor did they provide detailed explanations for missing data handling or report model calibration plots or relevant statistical tests.

Four studies (15, 25, 26, 39) demonstrated comprehensive performance across all evaluation domains. In these 4 studies, the recommended EPV was ≥10. Ruhua Zheng (2020) and Todd Brenner (2025) employed multiple imputation methods, while Youming Xu (2024) utilized a complete-case analysis. All four studies reported external validation results and calibration performance. The model complexity was well matched to the sample size in each of these studies.

The meta-analysis assessed the performance of nine externally validated predictive models for PEP. As shown in Figure 4, The AUC values ranged from 0.63 to 0.97, with the Youming Xu (2024) model demonstrating the highest performance (AUC{{h}} {{/h}}={{h}} {{/h}}0.97 [0.93, 1.00]). Six models (15, 24, 26, 27, 30, 34) showed heterogeneity below 10%, indicating that their performance was relatively consistent across different validation sets. Three models (25, 39, 41) showed high heterogeneity, with I² values greater than 50%, including Chan Hyuk Park (2024) (I² = 61.25%), Ruhua Zheng (2020) (I² = 79.90%), and Masafumi Chiba (2021) (I² = 99.51%). The results of the z-test for all models were statistically significant (p < 0.05), indicating robust predictive capability during external validation.

We performed subgroup analysis based on model type and PEP diagnostic criteria. Both traditional models and machine learning models exhibited considerable heterogeneity across the subgroups (Figure 5). However, categorizing by PEP diagnostic criteria led to a notable reduction in heterogeneity (Figure 6).