Before extubating a mechanically ventilated patient, intensivists must evaluate the patient's risk of extubation failure (EF). This decision is usually based on the results of a rapid shallow breathing index (RSBI), which is most often assessed during the readiness evaluation to identify patients who may proceed with the spontaneous breathing trial (SBT), with either a T-piece or low-level pressure support (1–3). The RSBI is the ratio of respiratory frequency to tidal volume and is a commonly used weaning predictor (1–3). This method is not 100% predictive of extubation success (ES) by 72 h. The decision-making behind extubation is critically important, as failed extubation occurs in 10–20% of intensive care unit (ICU) patients (4, 5) and both delayed extubation as well as early extubation are associated with worse outcomes. Extubation delay is associated with ventilator-associated pneumonia (6, 7), increased length of stay, increased risk for downstream tracheostomy (8, 9) and increased mortality in brain-injured patients (8). Extubation failure after planned extubation is associated with adverse outcomes including increased hospital mortality, prolonged hospital stay, higher costs, and greater need for tracheotomy and transfer to post-acute care (10–13).

While there are numerous ventilator weaning predictors and types of SBTs (14, 15), there is a paucity of data on risk factors that predict EF prior to removal of the endotracheal tube. Predicting factors for ES and EF include amount of endotracheal secretions (8, 16, 17), cough strength (16, 18, 19), and mental status prior to extubation after a successful SBT (16, 20). Patients with moderate or abundant secretions have been 3–8 times more likely to fail extubation than those with few to no secretions (8, 19). Coplin et al. (16) stated that the Glasgow coma scale (GCS) score alone did not predict extubation outcome in brain-injured patients and should not be used to exclude extubation; however, other investigators reported that impaired mental status did predict EF (21, 22). Moreover, patients who fail extubation often retain carbon dioxide because of an imbalance among respiratory muscle strength and imposed load (1, 23–25). Patients extubated while developing hypercapnia (Paco₂ >45 mmHg) during a successful SBT may also have an increased risk of mortality due to respiratory failure compared to those who do not develop hypercapnia during SBT (2).

Therefore, we hypothesized that recurrent respiratory failure requiring reintubation after initial extubation could be estimated using a composite score of known risk factors available in the electronic medical record (EMR). Our main objective was to derive a simple clinical prediction tool using a multivariate model and validate the Re-Intubation Scale Calculation (RISC) score to predict respiratory failure requiring reintubation.

The 6,161 patients included in our study were randomly allocated into a derivation set (n = 3,080) and validation set (n = 3,081). In the derivation set, patients had a mean (SD) age of 61.7 (16.6) years, and 1,820 (59%) were men. Within 72 h, 393 patients (12.8%) experienced EF. Similarly, in the validation set, the mean (SD) age was 62.4 (16.6) years and 1,778 (58%) were men. Within 72 h, 353 patients (11.5%) experienced EF (Table 1). Patient endotracheal secretions, mechanical ventilation, and ICU admission diagnosis data for both sets are displayed in Appendices A–C.

Predictors of EF in the derivation set included underweight status (body mass index, <18.5 kg/m²; OR, 1.91; 95% CI, 1.12–3.26; P = 0.02), GCS score of ≥10 (OR, 1.68; 95% CI, 1.31–2.16; P < 0.001), mean airway pressure (MAP) closest to 1 min after SBT start within 15 min <10 cmH₂o (OR, 2.11; 95% CI, 1.68–2.66; P < 0.001), fluid balance of ≥1,500 mL 24 h prior to extubation (OR, 2.36; 95% CI, 1.87–2.96; P < 0.001), and mechanical ventilation ≥5 days (OR, 3.94; 95% CI, 3.04–5.11; P < 0.001) (Tables 2, 3). The derivation set had a C-index of 0.72 (95% CI, 0.70–0.75) (Figure 2).

Our logistic model in validation set demonstrated that as RISC increased by 1, the odds of having EF became 1.6-fold higher (OR, 1.58; 95% CI, 1.47–1.69; P ≤ 0.001) (Table 4). Receiving operating curve analysis revealed the best cutoff for RISC was 4, which demonstrated a sensitivity of 0.80 and specificity of 0.54 with AUC of 0.67 (95% CI, 0.65–0.69) (Appendix C). Calibration plot of observed vs. predicted EF in the validation set is displayed in Figure 3. Using the above model, the validation set had a C-index of 0.72 (95% CI, 0.69–0.75). The RISC score ranged from 0 to 8 with a median of 4 (Figure 4).

We successfully developed a multivariable RISC score to predict extubation failure after a successful SBT with readily available bedside predictors. The RISC score predicts extubation failure with the best cut-off at ≥4 demonstrating a sensitivity of 0.80 and a specificity of 0.54 with AUC of 0.67. This is a modest value to determine extubation failure. RISC score provides several multivariate risk factors that can be externally validated in future deep learning and machine learning predictive models. We acknowledge this model is limited and some would argue that a clinically useful tool should have a higher AUC; however, external testing with larger data sets may be helpful in reproducing these results.

Neurologic impairment was found to be a possible risk factor for EF in our study and was validated previously in several studies (16, 19, 37–39). Mokhlesi et al. (37) verified that a moderate GCS score (9–12) can clinically predict reintubation, comparable to our results. We feel this is an important finding for clinicians caring for those patients with an “intermediate” GCS in their decision-making for extubation. GCS is inherently limited in finding lower cutoffs since a “V1-NT” is the subcomponent reported in intubated patients. Moreover, its components only grade Eyes (E1-4) and Motor (M1-6) which if maximal total 10 points (E4+M6), similar to our observed lowest cutoff. Some suggest using the Full Outline of UnResponsiveness (FOUR) score (40) might provide a more granular range (0–16) by including brainstem cranial nerve findings. We reviewed our data for FOUR score (40), but only had 228 patients, which is statistically underpowered to detect lower limits of this coma scale for reintubation risk. Only a future study using FOUR score with sufficient sample size may be able to detect this as a true reintubation risk, or in a future randomized trial in brain-injured patients similar to the one performed by WDF, which previously showed no difference in reintubation rates in GCS 10 or less patients with intact brainstem protective reflexes of cough/gag (9). We find documentation of protective airway or cranial nerve reflexes lacking in most and/or all coma scales, even in the FOUR score which focuses on pupillary reflexes for the brainstem. Furthermore, the FOUR score did not include gag/cough reflexes in the randomized trial by Manno et al. (9).

Another covariate of the RISC score is a MAP closest to 1 min after SBT start within 15 min <10 cmH₂o. MAP is dependent on peak inspiratory pressure, PEEP, and respiratory cycle time. Our finding of low MAP leading to extubation failure may be explained by a dyspneic patient with vigorous efforts. Mean airway pressure is a pressure monitoring metric used by mechanical ventilators that are closely connected to mean alveolar pressure and depict pressures on the lung parenchyma during ventilation (41). It's also connected to the oxygenation index (42). Peak inspiratory pressure, PEEP, and the inspiratory-to-expiratory time ratio with dynamic and real-time features are used to calculate MAP, which measures mechanical power impacted by the ventilator mode (43). A high MAP suggests that the patient's mechanical energy power is greater (43). Furthermore, MAP is a critical pressure parameter that influences a patient's hemodynamics. It has been established that higher MAP causes a reduction in cardiac output in infants during both normal and high-frequency mechanical ventilation (44, 45). In direct proportion to their effects on MAP, tidal forces and PEEP raise pulmonary vascular resistance (PVR) (46). As a result, MAP is becoming more linked to the prognosis of patients on mechanical ventilation. A patient that is not able to mount a high MAP may demonstrate insufficient respiratory mechanics from possibly, diaphragmatic weakness. Patients who are able to mount a high MAP may have more strength in respiratory muscles, and thus mechanics, thereby resulting in tolerance of extubation. In summary, we argue that MAP has several strengths over other conventional mechanical ventilator parameters especially pressure indicators, and it has the ability to become a major bridge factor coupling respiratory mechanics and hemodynamics. However, we acknowledge there are factors potentially not assessed in this study during the transition from positive to negative pressure physiology, stressors on cardiac physiology, and potentially Co₂ accumulation that could theoretically occur before extubation.

Thille et al. (33) identified prolonged mechanical ventilation (duration <1 week) prior to weaning as a strong predictor for EF, similar to our results. We observed that underweight status was also associated with EF and an important factor since it underlies neuromuscular reserve. Malnutrition is reported in as much as 40% of critically ill patients, but data linking nutritional status to ventilator weaning issues are limited (47, 48). In the acute respiratory distress syndrome (ARDS) network trial (43); 4.7% of patients were identified as underweight. Underweight patients, defined as those with a body mass index <18.5 kg/m², may experience depressed ventilatory drive (49), limited muscle mass (50), and weaning difficulty (51). Obesity may be associated with a better prognosis (“obesity paradox”) for some disease states, patients with ARDS (52) and those in the ICU in general (53, 54). Another important covariate in EF was positive fluid balance, as reported by Frutos-Vivar et al. (27). This is an important finding for clinicians to review prior to extubation attempts since diuretics could be given in future RISC-driven randomized trials on EF. D'Orio et al. (55) reported that a positive cumulative fluid balance may cause increased capillary leak and extravascular lung water and decreased lung compliance, leading to respiratory failure, both during SBT and in the immediate post-extubation period. Hence why restrictive fluid strategies are employed in ARDS patients (56). In an earlier study, positive cumulative fluid balance from hospital admission to weaning was correlated with EF (57). Therefore, it is highly plausible that positive fluid balance influences the respiratory outcome of patients.

To our knowledge, there are no comprehensive data sets analyzing all the variables we proposed in predicting ES, particularly with neurologic components. Most studies focus on RSBI as a predictor in the post-operative setting (27, 37, 58). RSBI is challenging however in patients in pain with tachypnea and some neurologic patients with abnormal brain-disordered breathing states. The closest study we found in the literature similar to ours was a neonatal extubation modeling study by Mueller et al. (54), who used artificial neural-networks, receiver operating characteristics, and regression modeling to predict ES. The authors looked at inspiratory to expiratory ratio, inspiratory time, MAP, tidal volume, and Sao₂. The authors found an AUC of 0.87, which is a fairly strong prediction for ES (59). Other studies have studied ES but these have been largely focused on operating room predictors and not global predictors (60). Rodriguez Blanco et al. (60) studied 78 surgery patients who had adequate ejection fraction and other standard clinical factors. This study did not adequately characterize the respiratory variables proposed and was of relatively small sample size. Similarly, data from a cohort of mechanically ventilated elderly patients were prospectively analyzed and used to develop a predictive model using a classification and regressive tree (CART) algorithm, also known as a decision tree to predict extubation outcome in patients following a successful SBT (29, 30). This CART model showed a good discrimination with an AUC of 0.94. However, calibration was moderate with a substantial mismatch between predicted and actual probabilities in the updated CART model (47).

Our study has several limitations. First, the retrospective design limits extrapolation to prospective and individualized patient care contexts. Retrospective studies can introduce selection and information biases. We were also not able to collect data on all clinical weaning predictors such as diaphragm movement, endotracheal secretions, and hypercapnic ventilatory response, Airway occlusion pressure in each patient. Second, our study lacked granular data on supplemental oxygen strategies post-extubation as recent literature supports high flow nasal cannula combined with non-invasive in preventing re-intubation (61, 62). Third, the study was completed at a single center with retrospective data collection, and thus does not allow inferences on causality derived from prospective data. Fourth, we were not able to collect echocardiographic measurements to be able to correlate positive fluid balance with ventricular dysfunction. Fifth, in our model, there may be some lack of variability using a retrospective cohort from the same center, and thus validation and accuracy may be overestimated. Sixth, some of the ventilator data captured in this study such as the reporting of a plateau pressure measurement during pressure support in a spontaneously breathing patient were from Puritan Bennett 840 ventilators (Medtronic). These ventilators record into the EMR what it believes are plateau pressures in any mode, even when an adequate inspiratory pause is absent. We have documented instances where vitals are being recorded at greater than the 15-min standard of the time for procedures or more while ventilator settings remained as they were. Lastly, our multivariate covariates within our RISC score model are generally known single risk factors of EF in the existing literature (27, 28, 33–35, 63). Therefore, our study should be considered with caution as exploratory only and requires prospective and external validation of the multivariate model before implementing into routine clinical decision making. This is especially true given data is from a single site that may reflect a unique culture not practiced by others.

Despite our limitations, this study has several strengths. A major strength is that it was done using a robust and clean dataset derived from a previously validated database (64). The study also included a large cohort of patients from a large, tertiary academic hospital. To date, there are no comprehensive data sets analyzing all the variables in predicting ES, especially with neurological components; therefore, we feel this study could represent the first adult human modeling study, and by the addition of more variables, an even more potentially accurate and precise model for predicting ES in the future. However, we still recommend external validation before generalizing and implementing these results as predictive models.

We developed the RISC score using several practical clinical parameters tested within a derivation and validation set. This model provides risk-stratification for extubation and subsequent EF within 72 h in mechanically ventilated patients in the ICU. Overall, we identified 5 predictors of EF readily available at the bedside and in many EMRs: underweight status (body mass index <18.5 kg/m²), GCS score ≥10, MAP closest to 1 min after SBT start within 15 min <10 cmH₂o, fluid balance of ≥1,500 mL 24 h prior to extubation, and mechanical ventilation ≥5 days. External validation in a larger, multicenter study is required before clinical implementation.

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

The studies involving human participants were reviewed and approved by Mayo Clinic Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Presented at the Neurocritical Care Society 15th Annual Meeting, Waikoloa Village, Hawaii, October 13-17, 2017 and portions of this manuscript have been published in abstract form and available online at https://link.springer.com/journal/12028/volumes-and-issues/27-2/supplement and citation “Bansal V, Li Z, Marquez A, Kashyap R, Smelick CP, Diaz-Gomez JL, et al. Re-Intubation Scale Calculation (RISC): predicting extubation failure in critically ill patients. Neurocritical Care. (2017) 27(Suppl. 2):S122.”

VB, NS, RK, and WF contributed to the conception and design of the study. VB and AM organized the database. VB and ZL performed the statistical analysis. All authors contributed to the article and approved the submitted version.

This work was supported by the Department of Neurology at Mayo Clinic, Jacksonville, Florida with no direct funding.

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.