Endotracheal intubation (EI) is a critical life-sustaining procedure for patients experiencing respiratory distress. Traditionally, EI decisions are made based on clinical assessment and available physiologic indicators, including vital signs and clinical scoring systems (1–3). Timely and accurate identification of newborns who are likely to require EI remains challenging in clinical practice because their clinical presentations and physiologic conditions can vary widely. In addition, delayed or inappropriate airway management in newborns with worsening respiratory failure can be associated with serious complications, including hypoxemia and bradycardia (4–6). Therefore, given these concerns, objective and quantifiable decision-support tools are needed to facilitate EI decision-making.

Recent studies have demonstrated that artificial intelligence (AI) models can support physicians in decision-making by predicting the optimal timing for EI based on patient data (7–10). To predict the need for EI at the time of patient admission, Nopour et al. (7) and Yu et al. (8) developed models based on multilayer perceptron (MLP) and eXtreme Gradient Boosting (XGBoost), respectively. Arvind et al. (9) utilized a random forest algorithm to predict EI requirements by analyzing time-series data, including blood pressure, respiratory rate, and oxygen saturation, at 12-h intervals over 3 days. Siu et al. (10) not only predicted the necessity of EI within 24 h for patients with respiratory distress but also identified key contributing variables in the model's decision-making process using Shapley additive explanations. Furthermore, Jueng-Eun et al. (11, 12) proposed AI models that can predict the timing of EI in 1-h intervals up to 12 h in advance. Their models leveraged initial clinical data from neonatal intensive care units (NICUs) along with hourly vital sign measurements for neonates experiencing respiratory distress.

However, previous studies (7–12) have three limitations. First, some studies (7, 8) do not reflect the health status of the patient over time. Physicians typically assess the condition of a patient by monitoring variations in vital signs, such as heart rate and oxygen saturation; however, these studies relied solely on initial clinical data obtained at the time of admission, without incorporating temporal changes in patient health. Second, the prediction time intervals set by previous studies (9–12) are limited. As delayed EI causes adverse effects, the prediction interval for EI should be short with the capability of long-term predictions. Nonetheless, previous studies (9, 10) either predicted EI application in 12-h intervals or began the prediction only 12 h after NICU admission. Moreover, although NICU patients are often hospitalized for extended periods, some studies (11, 12) only predicted EI application within the first 12 h after admission. Finally, existing models (7–10) may not be applicable to neonatal patients. Neonates exhibit distinct physiological characteristics in their respiratory and circulatory systems compared to adults (13, 14). As a result, EI prediction models originally designed for adult patients may not be suitable for neonatal use. Therefore, AI models should be developed to enable real-time assessment of patient health status, incorporate clinically relevant prediction intervals, and account for the unique physiological characteristics of neonates.

To address these limitations, we developed a multimodal model that integrates two types of data: initial clinical data (numerical data) and vital signs (time-series data). An overview of the proposed workflow is shown in Figure 1. The model utilizes a multilayer perceptron (MLP) to process numerical data and a long short-term memory (LSTM)–transformer architecture to analyze time-series data. LSTM–Transformer is designed to capture micro-dynamics and inter-signal dependencies. To evaluate the model's performance and generalizability, we trained and internally validated it using a dataset of 432 patients from our institution, Chungbuk National University Hospital (CBNUH). External validation was conducted using a separate dataset of 176 patients from Chungnam National University Hospital (CNUH). Additionally, we extended the prediction window to 72 h after admission and enabled EI prediction at hourly intervals up to 3 h in advance, providing sufficient time for clinical preparation.

In summary, the contributions of our research are as follows:

The proposed model was compared with other models used in recent EI applied prediction studies [random forest (9, 10), logistic regression (7, 10), and MLP (7)], other machine learning (ML) models (light gradient boosted machine (LightGBM)), and multimodal models similar to the proposed model [MLP-LSTM (18, 19) and MLP-transformer (12, 20)]. ML models (Random Forest, Logistic Regression, and LightGBM) were implemented using the scikit-learn library and optimized via grid search. The grid search included tuning hyperparameters such as the number of estimators for Random Forest, the regularization strength for Logistic Regression, and the learning rate and number of leaves for LightGBM. Deep learning models (MLP, MLP-LSTM, and MLP–transformer) were trained on numeric and time-series data and optimized via randomized search. The performances of each model are summarized in Table 2. As shown in Table 2 and Figure 5a, the proposed model achieved the following performances in the four averaged metrics: accuracy = 0.9579 ± 0.02, sensitivity = 0.8472 ± 0.08; specificity = 0.9591 ± 0.02; AUROC = 0.9323 ± 0.03. The proposed model showed the best performance in all evaluation metrics as compared to those of previous studies (7, 9, 10) and ML models. Furthermore, when comparing the evaluation metrics of the proposed model with other models based on multimodal architecture (12, 18–20), the proposed model demonstrates the best results in terms of accuracy, specificity, and AUROC, as well as the second-best performance in sensitivity. The MLP-LSTM model performed second best, with a difference of within 2% for each performance metric compared to the proposed model. Most models showed low performance in sensitivity and were weak at predicting positive cases.

To validate the generalization of the proposed model, external validation was performed using datasets collected from other institutions. The results were summarized using a confusion matrix and ROC curve as shown in Figure 5b. We compared the external validation results of the proposed model with those of other models, and the results of the evaluation metrics are listed in Table 2, while comparative results of the ROC curve are shown in Figure 6b. As listed in Table 2, the proposed model achieved the following performances in the four average metrics on the external dataset: accuracy = 0.9411 ± 0.01; sensitivity = 0.6739 ± 0.08; specificity = 0.9416 ± 0.01; AUROC = 0.9336 ± 0.02. The proposed model demonstrated good performance in accuracy, specificity and AUROC when compared with previous studies (7, 9, 10) and ML models in external validation. Logistic regression and LightGBM showed sensitivity 0.8913 ± 0.08 and 0.7283 ± 0.22, respectively, in the external validation results. However, they exhibited a bias toward positive cases predictions attributed to their low accuracy and specificity. The proposed model achieved the third-highest sensitivity, and excluding the two models that exhibited a bias toward positive case predictions, it showed the highest sensitivity among the remaining models. Moreover, we compared the evaluation metrics of the proposed model with other multimodal-based models (12, 18–20) and found that it performs well on all evaluation metrics. Deep learning-based models (MLP, MLP-LSTM, MLP-transformer, and proposed model) demonstrated good performance in accuracy, specificity, and AUROC, but exhibited relatively low values in sensitivity.

Figure 6 shows the ROC curves comparing the proposed model with other baseline models. The proposed model showed superior AUC and ROC curve performance in both internal and external validation. Furthermore, in Figure 7, to compare the classification results between the proposed model and the baseline models, we performed a non-parametric permutation test of the ROC curves (26). In internal validation(in Figure 7a), all models achieved similar AUROC values (0.91—0.93) with no statistically significant difference (p-value>0.05). This suggests that most models could effectively learn the underlying patterns within the same institutional dataset. In contrast, in the external validation(in Figure 7b) using data from a different institution, the proposed model demonstrated a statistically significant improvement (p-value < 0.05) compared to other baselines. Despite differences in data distribution and measurement frequency, the model maintained high AUROC (0.93) and accuracy (0.94), indicating strong generalization performance under domain shift conditions. This robustness suggests the model's potential for multi-institutional deployment. This finding implies that the hybrid LSTM–Transformer architecture effectively captured robust temporal dependencies and cross-feature interactions, which conventional models such as random forest or MLP-based approaches failed to generalize.

However, as shown in Table 2 and Figure 6, the model performance was lower in the external validation (sensitivity = 0.8472) as compared to that in the internal validation (sensitivity = 0.6739). This was because of two issues with the dataset collected for external validation. First, the external institution that collected the dataset used for external validation had a high number of missing positive cases owing to the institutional tendency of not recording vital sign data on an immediately before applying EI to a patient. This resulted in a significantly unbalanced dataset between positive and negative data (positive cases: 14, negative cases: 165). This can be addressed by focusing on collecting more positive cases during future data collection. Second, the datasets we collected had many missing RRs: the training dataset was collected at hourly intervals, while the external institution dataset was collected at irregular time intervals. Moreover, both internal and external validation results indicate that sensitivity showed lower performance compared to accuracy, specificity, and AUROC. This is the result of a trade-off between sensitivity and specificity, which can be resolved by adjusting the numbers.

In this study, the proposed model offers predictions of intubation probabilities using multimodal information including 3 h of vital signs and initial clinical variables. To develop and evaluate this model, we established a dataset including information on 432 neonatal patients and evaluated a multimodal model that could classify neonates requiring intubation 3 h in advance. The model used for validation was validated using fold 4, which showed the best performance metrics among the four trained folds. The AI model recommended EI if the probability exceeded 78.1% (optimal threshold of fold 4); otherwise, it did not recommend EI. In Figures 8, 9, the process of determining the probability of predicting EI, alongside the vital signs (HR, RR, and SpO₂) over time, is depicted based on the data of six random patients using the proposed model is shown. Figure 8 illustrate the intubation cases where EI was applied either 3, 5, or 6 h after admission, respectively. Figure 9 show the non-intubation case. Figure 8a shows that from 0 to 3 h after admission, HR increased by approximately 7.64%, RR decreased by approximately 15.79%, and SpO₂ showed minimal change in value. The probability of EI occurring during this period varied from 78% to 97%. As evident from Figure 8b, the RR decreased sharply to 51.6% between the 2 and 3 h time of admission, and the change gradually stabilized from 3 to 5 h; therefore, the probability of applying EI went down reduced by approximately 5.5%. Furthermore, during at 2–3 h after admission, the RRs was 80 and 74 breaths per min, respectively, which were out of the normal range (normal range of RR: 30–60 breaths per min). In Figure 8c, HR decreased by 12.8% between 4 and 6 h to 109–110 beats per min, which was above the normal range (normal range of HR: 120–160 beats per min). From 4–6 h, the HR remained low, and the probability of EI increased from 63.2% to 98.6%. Figure 9 show that the HR and RR values often appeared to deviate from the normal range. Furthermore, the vital signs changed rapidly and the probability of applying EI increased or decreased. However, we confirmed that the predictive probability of intubation was stable. This appears to reflect the analysis results of numeric data because our model integrates numeric and time-series data. Figures 8, 9 were designed not only to visualize prediction dynamics but also to provide case-based interpretability, showing that the predicted EI probability follows clinically plausible vital sign trajectories.

The proposed model has three primary limitations in analyzing time-series data.

First, it exhibits bias when predicting short sequences. The model processes sequences up to 3 h in 1-h increments. Among the positive cases collected, 31.6% (31/98) underwent EI within 2 h of NICU admission. Consequently, 31.3% of patients had missing data due to backfilling, which may have introduced bias in the analysis. Therefore, further research is needed to develop a model specifically designed to analyze short-sequence vital signs data and integrate it into the existing framework. Accordingly, predictions generated immediately after NICU admission, when substantial backfilling may be required, should be interpreted with caution.

Second, the proposed model heavily relies on time-series data. The dataset was augmented in 1-h intervals, and the last 3 h of data were extracted. While the proposed model utilizes separate encoders for static and time-series inputs, the window-based augmentation replicates static variables across multiple time steps, which may increase the model's reliance on time-series patterns. Future work will explore fusion strategies (e.g., modality gating or modality-specific regularization) to better balance contributions from static and temporal features.

Third, to preserve the intrinsic temporal dependencies and physiological consistency of the time-series data, no synthetic oversampling techniques (e.g., SMOTE) were applied, even though there was a notable class imbalance between the intubated (positive) and non-intubated (negative) groups. Instead, the imbalance was indirectly compensated by applying a weighted binary cross-entropy loss during model training, which penalized misclassification of the minority (intubated) class more heavily. In addition, while we report performance using an optimal threshold determined by the Youden index, the operating threshold can be calibrated to prioritize sensitivity for early warning applications depending on institutional clinical priorities. Regarding external validation, the reduced sensitivity observed in external validation may reflect a limited number of positive cases and outcome-dependent missingness (e.g., incomplete near-event vital-sign capture) as well as irregular charting intervals (particularly for RR) at the external site. Future external validation should adopt prospective multi-center protocols that ensure comprehensive vital-sign recording immediately preceding EI and investigate approaches robust to irregular sampling (e.g., time-aware model inputs and enhanced interpolation methods).

We developed a real-time prediction system utilizing a multimodal model to predict neonatal EI 3 h in advance. The proposed model outperformed previously developed models, achieving the following results: accuracy = 0.9579 ± 0.02, sensitivity = 0.8472 ± 0.08, specificity = 0.9591 ± 0.01, and AUROC = 0.9323 ± 0.03. The proposed model also demonstrated superior performance in external validation using datasets collected from an independent institution, achieving: accuracy = 0.9411 ± 0.01, sensitivity = 0.6739 ± 0.08, specificity = 0.9416 ± 0.01, and AUROC = 0.9336 ± 0.02. Moreover, the model was designed to reflect real-world clinical applications by incorporating three prediction windows: hourly intervals, within 3 h, and up to 72 h after admission. We anticipate that the proposed model will contribute to the early prevention of respiratory distress by enabling timely EI predictions in neonates. Given its generalizability and effectiveness, the model holds significant potential for clinical implementation. In future clinical implementation, the model could be integrated into real-time NICU monitoring systems to provide early warnings to physicians when the predicted probability of EI exceeds the predefined threshold. This could allow pre-intubation preparation, reduce procedure-related complications, and improve neonatal outcomes.