<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Med.</journal-id>
<journal-title>Frontiers in Medicine</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Med.</abbrev-journal-title>
<issn pub-type="epub">2296-858X</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.3389/fmed.2021.789874</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Medicine</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Prediction and Clinically Important Factors of Acute Kidney Injury Non-recovery</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name><surname>Liu</surname> <given-names>Chien-Liang</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/1558544/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Tain</surname> <given-names>You-Lin</given-names></name>
<xref ref-type="aff" rid="aff2"><sup>2</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/285515/overview"/>
</contrib>
<contrib contrib-type="author">
<name><surname>Lin</surname> <given-names>Yun-Chun</given-names></name>
<xref ref-type="aff" rid="aff1"><sup>1</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/1507413/overview"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name><surname>Hsu</surname> <given-names>Chien-Ning</given-names></name>
<xref ref-type="aff" rid="aff3"><sup>3</sup></xref>
<xref ref-type="aff" rid="aff4"><sup>4</sup></xref>
<xref ref-type="corresp" rid="c001"><sup>&#x0002A;</sup></xref>
<uri xlink:href="http://loop.frontiersin.org/people/537700/overview"/>
</contrib>
</contrib-group>
<aff id="aff1"><sup>1</sup><institution>Department of Industrial Engineering and Management, National Yang Min Chiao Tung University</institution>, <addr-line>Hsinchu</addr-line>, <country>Taiwan</country></aff>
<aff id="aff2"><sup>2</sup><institution>Department of Pediatrics, Kaohsiung Chang Gung Memorial Hospital and College of Medicine, Chang Gung University</institution>, <addr-line>Kaohsiung</addr-line>, <country>Taiwan</country></aff>
<aff id="aff3"><sup>3</sup><institution>Department of Pharmacy, Kaohsiung Chang Gung Memorial Hospital</institution>, <addr-line>Kaohsiung</addr-line>, <country>Taiwan</country></aff>
<aff id="aff4"><sup>4</sup><institution>School of Pharmacy, Kaohsiung Medical University</institution>, <addr-line>Kaohsiung</addr-line>, <country>Taiwan</country></aff>
<author-notes>
<fn fn-type="edited-by"><p>Edited by: Zaid A. Abassi, Technion Israel Institute of Technology, Israel</p></fn>
<fn fn-type="edited-by"><p>Reviewed by: Yuri Gorelik, Rambam Health Care Campus, Israel; Foroud Shahbazi, Kermanshah University of Medical Sciences, Iran; Joel Neugarten, Montefiore Health System, United States</p></fn>
<corresp id="c001">&#x0002A;Correspondence: Chien-Ning Hsu <email>chien_ning_hsu&#x00040;hotmail.com</email></corresp>
<fn fn-type="other" id="fn001"><p>This article was submitted to Nephrology, a section of the journal Frontiers in Medicine</p></fn></author-notes>
<pub-date pub-type="epub">
<day>17</day>
<month>01</month>
<year>2022</year>
</pub-date>
<pub-date pub-type="collection">
<year>2021</year>
</pub-date>
<volume>8</volume>
<elocation-id>789874</elocation-id>
<history>
<date date-type="received">
<day>05</day>
<month>10</month>
<year>2021</year>
</date>
<date date-type="accepted">
<day>10</day>
<month>12</month>
<year>2021</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#x000A9; 2022 Liu, Tain, Lin and Hsu.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Liu, Tain, Lin and Hsu</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/"><p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</p></license> </permissions>
<abstract>
<sec>
<title>Objective</title>
<p>This study aimed to identify phenotypic clinical features associated with acute kidney injury (AKI) to predict non-recovery from AKI at hospital discharge using electronic health record data.</p></sec>
<sec>
<title>Methods</title>
<p>Data for hospitalized patients in the AKI Recovery Evaluation Study were derived from a large healthcare delivery system in Taiwan between January 2011 and December 2017. Living patients with AKI non-recovery were used to derive and validate multiple predictive models. In total, 64 candidates variables, such as demographic characteristics, comorbidities, healthcare services utilization, laboratory values, and nephrotoxic medication use, were measured within 1 year before the index admission and during hospitalization for AKI.</p></sec>
<sec>
<title>Results</title>
<p>Among the top 20 important features in the predictive model, 8 features had a positive effect on AKI non-recovery prediction: AKI during hospitalization, serum creatinine (SCr) level at admission, receipt of dialysis during hospitalization, baseline comorbidity of cancer, AKI at admission, baseline lymphocyte count, baseline potassium, and low-density lipoprotein cholesterol levels. The predicted AKI non-recovery risk model using the eXtreme Gradient Boosting (XGBoost) algorithm achieved an area under the receiver operating characteristic (AUROC) curve statistic of 0.807, discrimination with a sensitivity of 0.724, and a specificity of 0.738 in the temporal validation cohort.</p></sec>
<sec>
<title>Conclusion</title>
<p>The machine learning model approach can accurately predict AKI non-recovery using routinely collected health data in clinical practice. These results suggest that multifactorial risk factors are involved in AKI non-recovery, requiring patient-centered risk assessments and promotion of post-discharge AKI care to prevent AKI complications.</p></sec></abstract>
<kwd-group>
<kwd>acute kidney injury</kwd>
<kwd>kidney function recovery</kwd>
<kwd>machine learning</kwd>
<kwd>risk prediction</kwd>
<kwd>interpretability</kwd>
<kwd>electronic health records&#x02014;EHR</kwd>
</kwd-group>
<counts>
<fig-count count="2"/>
<table-count count="2"/>
<equation-count count="0"/>
<ref-count count="33"/>
<page-count count="8"/>
<word-count count="5929"/>
</counts>
</article-meta>
</front>
<body>
<sec sec-type="intro" id="s1">
<title>Introduction</title>
<p>Acute kidney injury (AKI) is associated with increased worsening long-term outcomes and direct medical costs (<xref ref-type="bibr" rid="B1">1</xref>&#x02013;<xref ref-type="bibr" rid="B4">4</xref>). The recovery rate at hospital discharge or within 90 days from AKI ranges from 39.3 to 76.2% (<xref ref-type="bibr" rid="B5">5</xref>&#x02013;<xref ref-type="bibr" rid="B7">7</xref>) and the recovery rate (free from dialysis therapy) of patients with AKI requiring dialysis at 90 and 365 days after dialysis initiation ranges from 47.7 to 56.6% (<xref ref-type="bibr" rid="B8">8</xref>). Timing of kidney function recovery after an AKI episode is associated with an increased risk of onset and long-term progression of chronic kidney disease and can affect survival rates (<xref ref-type="bibr" rid="B6">6</xref>&#x02013;<xref ref-type="bibr" rid="B10">10</xref>). There is a need to improve the risk prediction for AKI non-recovery, and the predictive model may serve as a risk stratification tool to assist in the development of an appropriate post-AKI care plan at a time of hospital discharge.</p>
<p>Risk predictive models of AKI non-recovery using traditional statistical methods have achieved a C-statistic or area under the receiver operating characteristic curve (AUROC) of 0.61&#x02013;0.76 in the validation models (<xref ref-type="bibr" rid="B11">11</xref>&#x02013;<xref ref-type="bibr" rid="B13">13</xref>). The variance in AKI non-recovery predictive performance may be associated with heterogeneous AKI cohort, the definition of AKI recovery, and variable length of follow-up. A recent study suggested that clinical experts have selected features that can successfully predict AKI non-recovery within 7 days with an AUROC of 0.879 in the multi-center hospitalized cohort (<xref ref-type="bibr" rid="B14">14</xref>). These models tended to focus on patients with AKI undergoing dialysis or with critical illnesses. Machine learning approach using routinely collected health data in practice has demonstrated a promise for predicting AKI development in outpatient (<xref ref-type="bibr" rid="B15">15</xref>) or inpatient settings (<xref ref-type="bibr" rid="B16">16</xref>, <xref ref-type="bibr" rid="B17">17</xref>). Currently, there is not a single validated machine learning model to identify the risk predictors of AKI non-recovery to establish supporting strategies for post-discharge AKI care. This study aimed to develop and validate a machine learning model for predicting AKI non-recovery at hospital discharge for the development of targeted interventions to prevent long-term adverse kidney outcomes.</p>
</sec>
<sec sec-type="methods" id="s2">
<title>Methods</title>
<sec>
<title>Study Population</title>
<p>We used electronic health record data from the AKI Recovery Evaluation Study for a hospitalized adult cohort between January 1, 2010, and December 31, 2017, from the network of Chang Gung Memorial Hospitals in Taiwan. This study used the Chang Gung Research Database described in previous articles (<xref ref-type="bibr" rid="B18">18</xref>). Briefly, the adult hospitalized cohort from the AKI Recovery Evaluation Study included hospitalized patients who had an AKI episode at admission (index_AKI) and/or during the hospitalization (hospital-acquired AKI, HA_AKI) for analyzing the kidney function recovery after AKI exposure (as shown in <xref ref-type="supplementary-material" rid="SM1">Supplementary Material</xref> and <xref ref-type="supplementary-material" rid="SM1">Supplementary Table 1</xref>).</p>
<p>This study was approved by the Institutional Review Board of Chang Gung Memorial Foundation at Taipei, Taiwan, which approved a waiver of patient consent due to the retrospective nature of the study and use of de-identified data (permit number: 201901312B0C502). All datasets used in this study were de-identified prior to being transferred to the study investigators. The study followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement and recommendations for reporting machine learning analyses in the clinical research for reporting predictive model development and validation (<xref ref-type="bibr" rid="B19">19</xref>, <xref ref-type="bibr" rid="B20">20</xref>).</p>
</sec>
<sec>
<title>Study Outcomes</title>
<p>The outcome of AKI non-recovery was defined as a serum creatinine (SCr) value obtained close to the date of discharge (during the index hospitalization or within 3 days following discharge) &#x0003E;1.5-fold of the baseline SCr value (within 3 months before AKI hospitalization) (<xref ref-type="bibr" rid="B21">21</xref>). Patients without SCr data during their hospital stay and within 3 days post-AKI discharge (it is common to have the first follow-up visit in outpatient setting) and who died during the hospitalization were excluded from the analysis. Details of pre-hospitalization baseline SCr value are presented in <xref ref-type="supplementary-material" rid="SM1">Supplementary Material</xref>/Definition of the AKI cohort section.</p>
</sec>
<sec>
<title>Candidate Features and Imputation of Missing Data</title>
<p>Primarily, 64 candidate predictor features were available prior to and during the index hospitalization for AKI based on a review of the literature and clinical experience (as shown in the <xref ref-type="supplementary-material" rid="SM1">Supplementary Material</xref>/Candidate features section). These predictor features were patient demographic and clinical characteristics, laboratory results, nephrotoxic medication, and healthcare services utilization in pre-hospitalization and during the hospitalization. To increase the generalizability of the predictive model for both critical and non-critical ill hospitalized patients, absence of laboratory results was determined by the groups of patients with and without the use of intensive care units (ICUs) services to fill in the missing feature&#x00027;s values with the medians of the corresponding feature in the group to obtain the best model performance and maintain both sensitivity and specificity above 70% (<xref ref-type="supplementary-material" rid="SM1">Supplementary Material</xref>/Missing data imputation section/<xref ref-type="supplementary-material" rid="SM1">Supplementary Tables 1, 3</xref>).</p>
</sec>
<sec>
<title>Feature Selection</title>
<p>The recursive feature elimination based on the random forest was employed to determine the number of features that should be included during model training from all candidate features (<italic>n</italic> = 63) by constructing the same model repeatedly using caret package in R and random forest as a basic model. The result indicated that the model with 20 features can achieve the best performance (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 1</xref>). We used XGBoost to train the model and determine the 20 most important features (<xref ref-type="table" rid="T1">Table 1</xref>). Details of feature selection are presented in the <xref ref-type="supplementary-material" rid="SM1">Supplementary Material</xref>/Feature selection. This was to reduce the model complexity and prevent the final predictive model from overfitting that enables clinical applications. The distribution of each variable between recovery and non-recovery AKI groups was compared with chi-square test for categorical data and independent <italic>t</italic>-test was for means of continuous data.</p>
<table-wrap position="float" id="T1">
<label>Table 1</label>
<caption><p>Top 20 important features for predicting acute kidney injury (AKI) non-recovery.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th valign="top" align="left"><bold>Selected features</bold></th>
<th valign="top" align="center" colspan="4" style="border-bottom: thin solid #000000;"><bold>Derivation cohort</bold></th>
<th valign="top" align="center" colspan="4" style="border-bottom: thin solid #000000;"><bold>Temporal validation cohort</bold></th>
</tr>
<tr>
<th/>
<th valign="top" align="center"><bold><italic>N</italic> (<italic>n</italic> &#x0003D; 8,600)</bold></th>
<th valign="top" align="center"><bold>Recovery (<italic>n</italic> &#x0003D; 4,729)</bold></th>
<th valign="top" align="center"><bold>Non-Recovery (<italic>n</italic> &#x0003D; 3,871)</bold></th>
<th valign="top" align="center"><bold><italic>P</italic> value</bold></th>
<th valign="top" align="center"><bold><italic>N</italic> (<italic>n</italic> &#x0003D; 2,866)</bold></th>
<th valign="top" align="center"><bold>Recovery (<italic>n</italic> &#x0003D; 1,580)</bold></th>
<th valign="top" align="center"><bold>Non-recovery (<italic>n</italic> &#x0003D; 1,286)</bold></th>
<th valign="top" align="center"><bold><italic>P</italic> value</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left">Age at index date, mean (SD)</td>
<td valign="top" align="center">8,600</td>
<td valign="top" align="center">66.53 (15.05)</td>
<td valign="top" align="center">64.39 (16.09)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">2,866</td>
<td valign="top" align="center">67.08 (15.35)</td>
<td valign="top" align="center">64.20 (15.92)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">HA-AKI, <italic>n</italic> (%)</td>
<td valign="top" align="center">1,257</td>
<td valign="top" align="center">356 (7.53)</td>
<td valign="top" align="center">901 (23.28)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">406</td>
<td valign="top" align="center">124 (7.85)</td>
<td valign="top" align="center">282 (21.93)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">Index AKI stage at index admission, <italic>n</italic> (%)</td>
<td/>
<td/>
<td/>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td/>
<td/>
<td/>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Stage 2</td>
<td valign="top" align="center">1,384</td>
<td valign="top" align="center">586 (12.39)</td>
<td valign="top" align="center">798 (20.61)</td>
<td valign="top" align="center">524</td>
<td valign="top" align="center">217 (13.73)</td>
<td valign="top" align="center">307 (23.87)</td>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Stage 3</td>
<td valign="top" align="center">3,961</td>
<td valign="top" align="center">2,595 (54.87)</td>
<td valign="top" align="center">1,366 (35.29)</td>
<td valign="top" align="center">1,205</td>
<td valign="top" align="center">780 (49.37)</td>
<td valign="top" align="center">425 (33.05)</td>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">Chalrson comorbidity index (&#x0003C;1 year before index admission), <italic>n</italic> (%)</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Chronic kidney disease</td>
<td valign="top" align="center">2,774</td>
<td valign="top" align="center">2,062 (43.60)</td>
<td valign="top" align="center">712 (18.39)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">893</td>
<td valign="top" align="center">648 (41.01)</td>
<td valign="top" align="center">245 (19.05)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Cancer</td>
<td valign="top" align="center">2,332</td>
<td valign="top" align="center">951 (20.11)</td>
<td valign="top" align="center">1,381 (35.68)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">784</td>
<td valign="top" align="center">331 (20.95)</td>
<td valign="top" align="center">453 (35.23)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">Baseline laboratory results (&#x0003C; = 7 days before index admission), mean (SD)</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Index_SCr, mg/dL</td>
<td valign="top" align="center">8,600</td>
<td valign="top" align="center">4.65 (3.19)</td>
<td valign="top" align="center">2.96 (2.25)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">2,866</td>
<td valign="top" align="center">4.23 (3.07)</td>
<td valign="top" align="center">2.89 (2.26)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Baseline_SCr, mg/dL</td>
<td valign="top" align="center">8,600</td>
<td valign="top" align="center">3.29 (2.91)</td>
<td valign="top" align="center">1.30 (1.13)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">2,866</td>
<td valign="top" align="center">2.91 (2.69)</td>
<td valign="top" align="center">1.26 (1.13)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Blood urea nitrogen (BUN), mg/dL</td>
<td valign="top" align="center">5,965</td>
<td valign="top" align="center">46.15 (30.19)</td>
<td valign="top" align="center">26.58 (20.69)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">1,938</td>
<td valign="top" align="center">43.17 (30.39)</td>
<td valign="top" align="center">26.14 (20.80)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Potassium (K), mEq/L</td>
<td valign="top" align="center">6,559</td>
<td valign="top" align="center">4.25 (0.82)</td>
<td valign="top" align="center">4.07 (0.72)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">2,081</td>
<td valign="top" align="center">4.26 (0.80)</td>
<td valign="top" align="center">4.00 (0.67)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Low density lipoprotein cholesterol (LDL), mg/dL</td>
<td valign="top" align="center">2,820</td>
<td valign="top" align="center">98.11 (31.86)</td>
<td valign="top" align="center">100.70 (31.20)</td>
<td valign="top" align="center">0.0351</td>
<td valign="top" align="center">1,080</td>
<td valign="top" align="center">99.06 (31.63)</td>
<td valign="top" align="center">96.00 (31.35)</td>
<td valign="top" align="center">0.1215</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Serum uric acid (SUA), mg/dL</td>
<td valign="top" align="center">3,444</td>
<td valign="top" align="center">7.49 (2.30)</td>
<td valign="top" align="center">6.82 (2.41)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">1,183</td>
<td valign="top" align="center">6.87 (2.33)</td>
<td valign="top" align="center">6.49 (2.52)</td>
<td valign="top" align="center">0.0093</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Calcium (Ca), mg/dL</td>
<td valign="top" align="center">4,260</td>
<td valign="top" align="center">8.68 (0.77)</td>
<td valign="top" align="center">8.54 (0.70)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">1,362</td>
<td valign="top" align="center">8.63 (0.74)</td>
<td valign="top" align="center">8.55 (0.76)</td>
<td valign="top" align="center">0.0548</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;C-reactive protein (CRP), mg/L</td>
<td valign="top" align="center">5,161</td>
<td valign="top" align="center">82.13 (81.26)</td>
<td valign="top" align="center">83.47 (78.11)</td>
<td valign="top" align="center">0.5495</td>
<td valign="top" align="center">1,840</td>
<td valign="top" align="center">79.99 (80.73)</td>
<td valign="top" align="center">84.96 (79.61)</td>
<td valign="top" align="center">0.1907</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Albumin, g/dL</td>
<td valign="top" align="center">5,283</td>
<td valign="top" align="center">3.16 (0.65)</td>
<td valign="top" align="center">2.93 (0.67)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">1,665</td>
<td valign="top" align="center">3.22 (0.65)</td>
<td valign="top" align="center">2.96 (0.66)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Erythrocyte sedimentation rate (ESR), mm/hr</td>
<td valign="top" align="center">305</td>
<td valign="top" align="center">50.11 (30.20)</td>
<td valign="top" align="center">42.06 (31.18)</td>
<td valign="top" align="center">0.0249</td>
<td valign="top" align="center">78</td>
<td valign="top" align="center">50.08 (33.40)</td>
<td valign="top" align="center">41.28 (31.09)</td>
<td valign="top" align="center">0.2317</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;White blood cell (WBC)x10<sup>9</sup>/L</td>
<td valign="top" align="center">7,760</td>
<td valign="top" align="center">9.30 (4.75)</td>
<td valign="top" align="center">8.98 (5.11)</td>
<td valign="top" align="center">0.0042</td>
<td valign="top" align="center">2,594</td>
<td valign="top" align="center">9.36 (4.81)</td>
<td valign="top" align="center">9.31 (5.21)</td>
<td valign="top" align="center">0.7851</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Lymphocyte count (LPC), %</td>
<td valign="top" align="center">7,573</td>
<td valign="top" align="center">14.78 (9.69)</td>
<td valign="top" align="center">15.32 (11.59)</td>
<td valign="top" align="center">0.0297</td>
<td valign="top" align="center">2,565</td>
<td valign="top" align="center">14.48 (9.71)</td>
<td valign="top" align="center">15.38 (11.29)</td>
<td valign="top" align="center">0.0339</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Neutrophil count (NPC),%</td>
<td valign="top" align="center">7,557</td>
<td valign="top" align="center">74.95 (12.67)</td>
<td valign="top" align="center">73.18 (14.88)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">2,552</td>
<td valign="top" align="center">76.02 (12.24)</td>
<td valign="top" align="center">73.97 (14.20)</td>
<td valign="top" align="center">0.0001</td>
</tr>
<tr>
<td valign="top" align="left">Use of health service</td>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
<td/>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Number of outpatient visits &#x0003C; = 3 months before index date, mean (SD)</td>
<td valign="top" align="center">7,723</td>
<td valign="top" align="center">5.79 (6.54)</td>
<td valign="top" align="center">5.06 (4.16)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">2,607</td>
<td valign="top" align="center">4.90 (4.63)</td>
<td valign="top" align="center">5.03 (4.08)</td>
<td valign="top" align="center">0.4551</td>
</tr>
<tr>
<td valign="top" align="left">&#x000A0;Dialysis during the hospitalization, <italic>n</italic> (%)</td>
<td valign="top" align="center">1,718</td>
<td valign="top" align="center">993 (21.00)</td>
<td valign="top" align="center">1397 (36.09)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
<td valign="top" align="center">453</td>
<td valign="top" align="center">276 (7.47)</td>
<td valign="top" align="center">177 (13.76)</td>
<td valign="top" align="center">&#x0003C;0.0001</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>p: chi-square test was performed for categorical data and independent t-test was for means of continuous data between recovery and non-recovery AKI groups</italic>.</p>
</table-wrap-foot>
</table-wrap>
<p>The selected 20 features were further trained in stepwise logistic regression and four machine learning classification algorithms, such as adaptive least absolute shrinkage and selection operator (LASSO), random forest, eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM) (<xref ref-type="fig" rid="F1">Figure 1</xref>; <xref ref-type="supplementary-material" rid="SM1">Supplementary Table 4</xref>).</p>
<fig id="F1" position="float">
<label>Figure 1</label>
<caption><p>Study analysis flow. LR, logistic regression; LASSO, least absolute shrinkage and selection operator; XGBoost, eXtreme Gradient Boosting; LightGBM, Light Gradient Boosting Machine.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fmed-08-789874-g0001.tif"/>
</fig>
</sec>
<sec>
<title>Development and Validation of the Predictive Model</title>
<p>A five-fold cross-validation was used to confirm the combination of features and the final model for the outcome predictive model. The predictive performance of model discrimination was analyzed with the AUROC. The cut-off point was determined based on the best Youden index (<xref ref-type="bibr" rid="B22">22</xref>), to determine sensitivity and specificity in the final model. The patients hospitalized from 2016 to 2017 were considered as the temporal validation cohort to validate the predictive performance and generalization of the best final predictive models (<xref ref-type="table" rid="T2">Table 2</xref>).</p>
<table-wrap position="float" id="T2">
<label>Table 2</label>
<caption><p>Summary of model comparisons for predicting AKI non-recovery in the derivation cohort.</p></caption>
<table frame="hsides" rules="groups">
<thead><tr>
<th/>
<th valign="top" align="center"><bold>Stepwise LR</bold></th>
<th valign="top" align="center"><bold>LASSO</bold></th>
<th valign="top" align="center"><bold>XGBoost</bold></th>
<th valign="top" align="center"><bold>LightGBM</bold></th>
<th valign="top" align="center"><bold>Random forest</bold></th>
</tr>
</thead>
<tbody>
<tr>
<td valign="top" align="left" colspan="6"><bold>Full model</bold></td>
</tr>
<tr>
<td valign="top" align="left">AUROC</td>
<td valign="top" align="center">0.790 &#x000B1; 0.014</td>
<td valign="top" align="center">0.788 &#x000B1; 0.018</td>
<td valign="top" align="center">0.805 &#x000B1; 0.019</td>
<td valign="top" align="center">0.801 &#x000B1; 0.010</td>
<td valign="top" align="center">0.787 &#x000B1; 0.015</td>
</tr>
<tr>
<td valign="top" align="left">Sensitivity</td>
<td valign="top" align="center">0.779 &#x000B1; 0.065</td>
<td valign="top" align="center">0.654 &#x000B1; 0.030</td>
<td valign="top" align="center">0.651 &#x000B1; 0.082</td>
<td valign="top" align="center">0.667 &#x000B1; 0.058</td>
<td valign="top" align="center">0.726 &#x000B1; 0.072</td>
</tr>
<tr>
<td valign="top" align="left">Specificity</td>
<td valign="top" align="center">0.657 &#x000B1; 0.061</td>
<td valign="top" align="center">0.779 &#x000B1; 0.037</td>
<td valign="top" align="center">0.803 &#x000B1; 0.099</td>
<td valign="top" align="center">0.783 &#x000B1; 0.053</td>
<td valign="top" align="center">0.750 &#x000B1; 0.081</td>
</tr>
<tr>
<td valign="top" align="left">Precision</td>
<td valign="top" align="center">0.652 &#x000B1; 0.028</td>
<td valign="top" align="center">0.784 &#x000B1; 0.028</td>
<td valign="top" align="center">0.811 &#x000B1; 0.062</td>
<td valign="top" align="center">0.750 &#x000B1; 0.009</td>
<td valign="top" align="center">0.784 &#x000B1; 0.040</td>
</tr>
<tr>
<td valign="top" align="left">F1_score</td>
<td valign="top" align="center">0.708 &#x000B1; 0.023</td>
<td valign="top" align="center">0.713 &#x000B1; 0.023</td>
<td valign="top" align="center">0.717 &#x000B1; 0.034</td>
<td valign="top" align="center">0.738 &#x000B1; 0.013</td>
<td valign="top" align="center">0.751 &#x000B1; 0.024</td>
</tr>
<tr>
<td valign="top" align="left" colspan="6"><bold>Top-20 model</bold></td>
</tr>
<tr>
<td valign="top" align="left">AUROC</td>
<td valign="top" align="center">0.787 &#x000B1; 0.015</td>
<td valign="top" align="center">0.787 &#x000B1; 0.016</td>
<td valign="top" align="center">0.808 &#x000B1; 0.015</td>
<td valign="top" align="center">0.798 &#x000B1; 0.005</td>
<td valign="top" align="center">0.787 &#x000B1; 0.015</td>
</tr>
<tr>
<td valign="top" align="left">Sensitivity</td>
<td valign="top" align="center">0.654 &#x000B1; 0.032</td>
<td valign="top" align="center">0.657 &#x000B1; 0.044</td>
<td valign="top" align="center">0.661 &#x000B1; 0.037</td>
<td valign="top" align="center">0.694 &#x000B1; 0.037</td>
<td valign="top" align="center">0.672 &#x000B1; 0.064</td>
</tr>
<tr>
<td valign="top" align="left">Specificity</td>
<td valign="top" align="center">0.779 &#x000B1; 0.049</td>
<td valign="top" align="center">0.775 &#x000B1; 0.062</td>
<td valign="top" align="center">0.796 &#x000B1; 0.050</td>
<td valign="top" align="center">0.748 &#x000B1; 0.029</td>
<td valign="top" align="center">0.791 &#x000B1; 0.063</td>
</tr>
<tr>
<td valign="top" align="left">Precision</td>
<td valign="top" align="center">0.785 &#x000B1; 0.035</td>
<td valign="top" align="center">0.784 &#x000B1; 0.040</td>
<td valign="top" align="center">0.800 &#x000B1; 0.031</td>
<td valign="top" align="center">0.790 &#x000B1; 0.007</td>
<td valign="top" align="center">0.800 &#x000B1; 0.035</td>
</tr>
<tr>
<td valign="top" align="left">F1_score</td>
<td valign="top" align="center">0.713 &#x000B1; 0.015</td>
<td valign="top" align="center">0.713 &#x000B1; 0.018</td>
<td valign="top" align="center">0.723 &#x000B1; 0.016</td>
<td valign="top" align="center">0.705 &#x000B1; 0.016</td>
<td valign="top" align="center">0.728 &#x000B1; 0.027</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<p><italic>LR, logistic regression; LASSO, least absolute shrinkage and selection operator; XGBoost, eXtreme Gradient Boosting; LightGBM, Light Gradient Boosting Machine; and AUROC, area under the receiver operating characteristic curve</italic>.</p>
<p><italic>XGBoost with top-20 feature predictive results in the temporal validation cohort yielded an AUROC of 0.807, a sensitivity of 0.724, and a specificity of 0.738 (with the cut-off point = 0.471)</italic>.</p>
</table-wrap-foot>
</table-wrap>
<p>SHapley Additive exPlanations (SHAP) values, feature importance, summary plot, and dependency plot were used to explain the final predictive model (<xref ref-type="bibr" rid="B23">23</xref>). The Local Interpretable Model-Agnostic Explanations (LIME) method with four scenarios (true positive, false positive, true negative, and false negative outcome prediction) were randomly applied to elaborate the feature importance ranking and marginal effect in individual patient prediction (as shown in <xref ref-type="supplementary-material" rid="SM1">Supplementary Material</xref>/Interpretation of the predictive model) (<xref ref-type="bibr" rid="B24">24</xref>). All experimental analyses were performed using R (version 4.1.2; R Foundation for Statistical Computing, Vienna, Austria) and Python software (version 3.7.3; Python Software Foundation, Wilmington, DE, USA). The details of package, function, and parameters used in this study are presented in the <xref ref-type="supplementary-material" rid="SM1">Supplementary Material</xref>/Derivation and validation of the predictive model section.</p>
</sec>
</sec>
<sec sec-type="results" id="s3">
<title>Results</title>
<sec>
<title>Characteristics of the Study Cohort</title>
<p>A total of 11,466 patients with AKI who survived at the index hospital discharge were included in the derivation and temporal validation cohorts (8,600 in the 2010&#x02013;2015 derivation cohort and 2,866 in the 2016&#x02013;2017 validation cohort) (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 1</xref>). All 63 candidate features are reported in <xref ref-type="supplementary-material" rid="SM1">Supplementary Table 2</xref>. The mean age of the study cohort ranged from 64 to 67 years in both the derivation and validation cohorts, and male patients (&#x0003E;50%) were predominantly observed in the cohorts. The mean baseline (pre-hospitalization) SCr value was 1.3 (&#x000B1;1.13) mg/dl in patients with AKI non-recovery in both the derivation and validation cohorts. Moreover, the baseline SCr values were 2.91 (&#x000B1;2.69) and 3.29 (&#x000B1;2.91) mg/dl in patients with AKI recovery in the derivation and validation cohorts, respectively. The rates of dialysis and ICU utilization during hospitalization were 19.98% and 15.8%, 30% and 27.7% in the derivation and validation cohorts, respectively (<xref ref-type="supplementary-material" rid="SM1">Supplementary Table 1</xref>).</p>
</sec>
<sec>
<title>Predicting AKI Non-recovery and Machine Learning Algorithm Comparisons</title>
<p>A similar proportion of AKI non-recovery (45%) at hospital discharge was observed in both the derivation and validation cohorts. Moreover, the 20 features identified (<xref ref-type="table" rid="T1">Table 1</xref>) by XGBoost to predict AKI non-recovery outcome had a higher AUROC (0.808, SD 0.015) in the five-fold cross-validation in the derivation models. This indicates that the selected 20 most important features to train the model can achieve almost the same performance as that using all variables (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 1</xref>; <xref ref-type="table" rid="T2">Table 2</xref>).</p>
<p>In terms of prediction performance in the derivation models with top 20 features (<xref ref-type="table" rid="T2">Table 2</xref>; <xref ref-type="supplementary-material" rid="SM1">Supplementary Table 3</xref>), the AUROC values of each algorithm were closed among stepwise logistic regression (0.787 &#x000B1; 0.015), least absolute shrinkage and selection operator (LASSO) (0.787 &#x000B1; 0.016), random forest (0.787 &#x000B1; 0.015), LightGBM (0.798 &#x000B1; 0.005), and XGBoost models (0.808 &#x000B1; 0.015). The model with the best prediction performance was the XGBoost model, with maximized sensitivity and specificity of 0.661 (&#x000B1;0.037) and 0.796 (&#x000B1;0.05), respectively (<xref ref-type="table" rid="T2">Table 2</xref>). When F1 score was examined, the random forest (0.728 &#x000B1; 0.027) had a slightly higher F1 score than XGBoost model (0.723 &#x000B1; 0.164) in <xref ref-type="table" rid="T2">Table 2</xref>. Based on AUROC, the XGBoost model was chosen to validate its predictive performance in the temporal validation cohort. The optimal cutoff value was 0.471 to achieve good discriminatory power for AKI non-recovery vs. AKI recovery outcomes with the AUROC, sensitivity, and specificity of 0.807, 0.724, and 0.738 in the validation cohort, respectively. In addition, precision value and F1 score of the XGBoost model in the validation cohort were 0.692 and 0.708, respectively.</p>
</sec>
<sec>
<title>Interpretation of the Predictive Model</title>
<p>The mean absolute SHAP value indicates individual feature importance in the XGBoost model (<xref ref-type="fig" rid="F2">Figure 2A</xref>). Overall, eight of the top 20 characteristics, such as AKI occurring during hospitalization (HA_AKI), SCr level at admission (index_SCr), receipt of AKI dialysis during hospitalization (index_DA_mod), baseline comorbidity of cancer, AKI at admission (index_AKI), baseline lymphocyte count (LPC%), and baseline potassium (K, mEq/L) and low-density lipoprotein cholesterol (mg/dl) levels, were sorted by decreasing importance in predicting AKI non-recovery outcomes (with deep pink color in <xref ref-type="fig" rid="F2">Figure 2A</xref>). The summary plot (<xref ref-type="fig" rid="F2">Figure 2B</xref>) shows that a low baseline SCr level increases the importance of non-recovery prediction, whereas a high baseline SCr level can greatly reduce the importance. Furthermore, SHAP dependency plot (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 2</xref>) demonstrates the interaction effect of two features on AKI recovery prediction.</p>
<fig id="F2" position="float">
<label>Figure 2</label>
<caption><p>SHapley Additive exPlanations (SHAP) results. <bold>(A)</bold> SHAP feature importance. Eight features with deep pink color had a positive effect on AKI non-recovery prediction, 11 features with blue color were negatively correlated with AKI non-recovery prediction (1 with neutral), and a higher mean SHAP value has a higher effect on outcome prediction. <bold>(B)</bold> SHAP summary plot. The <italic>x</italic>-axis denotes the SHAP value for each feature, whereas the color represents the feature value (<italic>y</italic>-axis) from high to low (deep pink to blue). SHAP, SHapley Additive exPlanations and AKI, acute kidney injury.</p></caption>
<graphic mimetype="image" mime-subtype="tiff" xlink:href="fmed-08-789874-g0002.tif"/>
</fig>
<p>The LIME visualization results of four random individual prediction scenarios are shown in <xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 3</xref>. <xref ref-type="supplementary-material" rid="SM1">Supplementary Figures 3A&#x02013;D</xref> provide an insight as to why the model makes a correct or incorrect prediction on an individual patient. Scenario A (ID 2231) and scenario B (ID2388) patients had similar characteristics. However, the scenario B patient had a higher index_SCr (1.99 mg/dl), baseline white blood cell (base_WBC) count (14 &#x000D7; 10<sup>9</sup>/L), baseline calcium (base_Ca) (&#x0003E; 8.5 mg/dl), and base_K (&#x0003E; 4.1 mEq/l) levels and a lower baseline C-reactive protein (base_CRP) level (30.76 mg/L) than the scenario A patient (SCr index = 0.98, WBC = 1.6 &#x000D7; 10<sup>9</sup>/l, base_Ca = 8.01 mg/dl, base_K = 3.1 mEq/l, and base_CRP = 166.72 mg/l). Feature importance ranking for the AKI recovery prediction is shown in <xref ref-type="supplementary-material" rid="SM1">Supplementary Figures 3C,D</xref>. High baseline SCr level (&#x0003E;3.54 mg/dl), without HA_AKI, and without prior cancer were the three most important features positivity correlated with AKI recovery prediction. Compared with the scenario C patient (a true AKI recovery prediction), the scenario D patient (a false AKI recovery prediction case) had a lower base_albumin (2.74 mg/dl), lower base_WBC count (4.8 &#x000D7; 10<sup>9</sup>/L), and lower SUA (5.1 mg/dl), but a higher calcium level (8.57 mg/dl) and higher number of nephrotoxic medication class used in outpatient setting (7 classes). These features could contribute false AKI recovery prediction in the scenario D.</p>
</sec>
</sec>
<sec sec-type="discussion" id="s4">
<title>Discussion</title>
<p>The present study results demonstrate an explainable machine learning model for predicting kidney function non-recovery at hospital discharge in patients receiving inpatient AKI care from routinely collected health record data in clinical practice. Timing of AKI occurrence, receipt of dialysis, and SCr measurements; comorbidity of cancer; and pre-hospitalization baseline LPC% and K levels, which are relevant for predicting AKI non-recovery, did not return to a level &#x02265;50% pre-hospitalization baseline SCr level. The SHAP and LIME methods enhance the interpretability of machine learning models and estimate the positive and negative contributions of each feature to the model prediction.</p>
<p>A previous study has identified four selected predictors (estimated glomerular filtration rate &#x02265; 30 ml/min/1.73 m<sup>2</sup>, preadmission hemoglobin level, platelet level, and diabetes comorbidity) of AKI recovery using stepwise logistic regression in patients with AKI undergoing dialysis therapy, however, failed to achieve acceptable discrimination performance (C-statistic, 0.645) 0.1 (<xref ref-type="bibr" rid="B3">3</xref>). Another study identified age, sex, SCr elevation, and urinary output as predictive factors for AKI recovery (without kidney replacement therapy within 5 days and SCr value &#x0003C;1.5 times the pre-ICU admission value) in patients requiring critical care, with an AUROC of 0.73 (65.4&#x02013;80.8%) 0.1 (<xref ref-type="bibr" rid="B1">1</xref>). However, these studies have generally included patients requiring dialysis therapy in ICUs and have been limited to severe forms of AKI. To the best of our knowledge, this is the first study to develop an AKI recovery predictive model that uses machine learning algorithms for patients with AKI who were and were not admitted to ICUs. Importantly, this study included baseline SCr values, which are commonly ignored in previous studies assessing the risk factor for AKI recovery (<xref ref-type="bibr" rid="B25">25</xref>, <xref ref-type="bibr" rid="B26">26</xref>). These previous studies have well-recognized limitations in the evaluation of AKI recovery. Based on the same data, our results showed that the XGBoost algorithm model achieved a good predictive performance with the 20 selected features and was reproducible in the validation cohort.</p>
<p>Another major contribution of this study includes its capability to derive reliable important features and validate an accurate AKI non-recovery predictive model based on a large AKI cohort with broad characteristic diversity. Given the poor outcomes among AKI survivors even in less severe AKI cases (<xref ref-type="bibr" rid="B3">3</xref>, <xref ref-type="bibr" rid="B7">7</xref>), these study results of AKI non-recovery predictors can facilitate risk assessment for identifying at-risk patients for poor kidney recovery before hospital discharge and promote nephrology referrals for post-discharge care for patients who may benefit most from such interventions (<xref ref-type="bibr" rid="B27">27</xref>, <xref ref-type="bibr" rid="B28">28</xref>).</p>
<p>Although the identified features should not be interpreted as etiologic factors or causal relationships, several temporal associations are noteworthy. These findings indicate that HA_AKI, high index SCr, and receipt of dialysis during the hospitalization were the three strongest predictors for non-recovery, a result consistent with those of prior studies (<xref ref-type="bibr" rid="B6">6</xref>, <xref ref-type="bibr" rid="B10">10</xref>, <xref ref-type="bibr" rid="B29">29</xref>). Because dialysis (intermittent and continuous hemodialysis) is a complementary therapy for AKI (<xref ref-type="bibr" rid="B30">30</xref>), it may be correlated with the severity of AKI. The fact that timing of AKI occurrence (AKI presence at admission [index_AKI] or during hospitalization [HA_AKI]) was also important to AKI non-recovery prediction, indicating that kidney function returned to the pre-hospitalization baseline level was not considered for hospital discharge in current practice. Further research is warranted to focus on long-term adverse outcomes among patients with AKI non-recovery who had a small improvement (e.g., recovery &#x0003C;50% baseline SCr) or worsening kidney function (e.g., increase &#x0003E; peak SCr) at hospital discharge.</p>
<p>The SHAP results showed that the baseline SCr value had bidirectional effects on AKI outcome prediction (<xref ref-type="fig" rid="F2">Figure 2B</xref>). It is possible that patients with a high baseline SCr value (proxy of poor kidney function) or less severe episode of AKI at admission may contribute less to AKI non-recovery prediction (negative SHAP value in <xref ref-type="fig" rid="F2">Figure 2B</xref>). In contrast, patients with a low baseline SCr value (blue color of feature value) may experience a severe episode of AKI at admission or during hospitalization, resulting in positive and significant effects on AKI non-recovery prediction (positive SHAP value in <xref ref-type="fig" rid="F2">Figure 2B</xref>).</p>
<p>Cancer is an important factor for predicting AKI non-recovery. These results are consistent that cancer patients with AKI non-recovery within 7 days had a greater risk of end-stage kidney disease or death (<xref ref-type="bibr" rid="B14">14</xref>). We hypothesized that existing cancer influencing AKI non-recovery may reflect higher baseline risk of AKI progression because patients with cancer are at high risk for infections, sepsis, tumor lysis syndrome, and chemotherapy-associated toxicities (<xref ref-type="bibr" rid="B31">31</xref>, <xref ref-type="bibr" rid="B32">32</xref>).</p>
<p>Although the relevance is low, the baseline lymphocyte count and low-density lipoprotein cholesterol level increased, positively predicting AKI non-recovery. Lymphocytes are associated with the systemic inflammatory response; the proportion of baseline LPC and its importance to AKI non-recovery prediction are not linear. Moreover, its relevance can be associated with hospital-acquired AKI (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 2A</xref>). These laboratory results suggested that inflammatory response at baseline could mediate the recovery phase of AKI (<xref ref-type="bibr" rid="B33">33</xref>). The SHAP dependence plot (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 2B</xref>) shows that baseline <italic>K</italic> value &#x0003E; 4 mEq/L increased the prediction of AKI non-recovery (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figure 2B</xref>). In contrast, patients with <italic>K</italic> &#x0003E; 5 mEq/L and high SCr value at admission had a negative association with AKI non-recovery prediction, which could be due to further medical interventions during the inpatient stay.</p>
<p>Despite the increasing interest in the relative predictive performance of different machine learning approaches, the results of this study are significant as they contribute to a better understanding of the effect (positive or negative correlation) of each feature of the predictive AKI non-recovery outcomes and a possible explanation of model&#x00027;s misclassification in the SHAP and LIME procedures. For instance, the LIME results potentially provided an in-depth explanation for why the scenario B patient (ID 2388), having a similar baseline SCr level with the scenario A patient (ID 2231 with &#x0003C; 1.0 mg/dl), was misclassified as AKI non-recovery (the scenario B patient recovered) in the final predictive model (<xref ref-type="supplementary-material" rid="SM1">Supplementary Figures 3A,B</xref>). All SHAP and LIME results suggested that to further perform an external validation in different populations, a study investigating the association between time-varying feature values (e.g., SCr trajectory) and mechanisms of AKI non-recovery should be conducted in the future.</p>
<p>This study has some limitations. First, only routinely available clinical data in the electronic health records were used to identify clinical phenotypes. Other data, such as urine output, fluid status, inpatient medication use, or pathogen variables during the hospitalization in the derivation model that could change classification, were not assessed. Additionally, considering the changes to the time window for AKI recovery, the measurable candidate feature inclusions may affect the selected features and outcome predictive performance. However, this does not undermine the current findings but rather highlights the need for further studies to better understand the complex mechanisms correlated with AKI non-recovery. Second, missing data were common for some laboratory results included in the derivation models. Although different approaches of data imputation were examined and the best calibration method was used to preserve at least 70% sensitivity and specificity, the imputed feature&#x00027;s value could bias classification. Finally, the characteristics of patient and clinical data were derived initially from a single integrated health system in Taiwan. Although feature distribution and frequency were found to be generalizable in the derivation and temporal validation cohorts, further external cohort validation is necessary, especially using data from different healthcare systems and more recent clinical data in academic center cohorts.</p>
<p>In this study, XGBoost demonstrated good discrimination performance for AKI non-recovery at hospital discharge. In the temporal validated model, 20 important patients and clinical characteristics reflect the complexity of factors affecting the non-recovery kidney function. The eight characteristics supporting AKI non-recovery prediction should be considered risk predictors of adverse AKI outcome in further risk profiling to drive the development of post-AKI care strategy for inpatients with AKI before hospital discharge.</p>
</sec>
<sec sec-type="data-availability" id="s5">
<title>Data Availability Statement</title>
<p>The original contributions presented in the study are included in the article/<xref ref-type="supplementary-material" rid="SM1">Supplementary Material</xref>, further inquiries can be directed to the corresponding author/s.</p>
</sec>
<sec id="s6">
<title>Ethics Statement</title>
<p>This study was approved by the Institutional Review Board of Chang Gung Memorial Foundation at Taipei, Taiwan. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.</p>
</sec>
<sec id="s7">
<title>Author Contributions</title>
<p>C-NH and Y-LT were involved in concept and design. C-NH and C-LL were involved in the drafting of the manuscript. C-LL and Y-CL performed statistical analysis. C-NH obtained funding. All the authors were involved in critical revision of the manuscript for important intellectual content.</p>
</sec>
<sec sec-type="funding-information" id="s8">
<title>Funding</title>
<p>This study was funded by Kaohsiung Chang Gung Memorial Hospital (CMRPG8K0101 and CORPG8L0041 to C-NH). The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.</p>
</sec>
<sec sec-type="COI-statement" id="conf1">
<title>Conflict of Interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="disclaimer" id="s9">
<title>Publisher&#x00027;s Note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
</body>
<back>
<ack><p>The authors thank Chun-Hua Liao and Ying-Jen Hsu from the Department of Management of Information Systems, Chang Gung Memorial Foundation, for their technical assistance with electronic health record acquisition and data retrieval. We would like to thank Ms. Chin-Yu Kuo in the Department of Industrial Engineering and Management, National Chiao Tung University (prior name) and Ms. Hsiao-Ching Kuo from the Department of Pharmacy of Kaohsiung Chang Gung Memorial Hospital for their assistance in the data cleaning process for their work on data management and machine learning programming.</p>
</ack>
<sec sec-type="supplementary-material" id="s10">
<title>Supplementary Material</title>
<p>The Supplementary Material for this article can be found online at: <ext-link ext-link-type="uri" xlink:href="https://www.frontiersin.org/articles/10.3389/fmed.2021.789874/full#supplementary-material">https://www.frontiersin.org/articles/10.3389/fmed.2021.789874/full#supplementary-material</ext-link></p>
<supplementary-material xlink:href="Data_Sheet_1.pdf" id="SM1" mimetype="application/pdf" xmlns:xlink="http://www.w3.org/1999/xlink"/>
</sec>
<ref-list>
<title>References</title>
<ref id="B1">
<label>1.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Coca</surname> <given-names>SG</given-names></name> <name><surname>Yusuf</surname> <given-names>B</given-names></name> <name><surname>Shlipak</surname> <given-names>MG</given-names></name> <name><surname>Garg</surname> <given-names>AX</given-names></name> <name><surname>Parikh</surname> <given-names>CR</given-names></name></person-group>. <article-title>Long-term risk of mortality and other adverse outcomes after acute kidney injury: a systematic review and meta-analysis</article-title>. <source>Am J Kid Dis Offic J Nat Kidney Found.</source> (<year>2009</year>) <volume>53</volume>:<fpage>961</fpage>&#x02013;<lpage>73</lpage>. <pub-id pub-id-type="doi">10.1053/j.ajkd.2008.11.034</pub-id><pub-id pub-id-type="pmid">19346042</pub-id></citation></ref>
<ref id="B2">
<label>2.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Chertow</surname> <given-names>GM</given-names></name> <name><surname>Burdick</surname> <given-names>E</given-names></name> <name><surname>Honour</surname> <given-names>M</given-names></name> <name><surname>Bonventre</surname> <given-names>JV</given-names></name> <name><surname>Bates</surname> <given-names>DW</given-names></name></person-group>. <article-title>Acute kidney injury, mortality, length of stay, and costs in hospitalized patients</article-title>. <source>J Am Soc Nephrol JASN.</source> (<year>2005</year>) <volume>16</volume>:<fpage>3365</fpage>&#x02013;<lpage>70</lpage>. <pub-id pub-id-type="doi">10.1681/ASN.2004090740</pub-id><pub-id pub-id-type="pmid">16177006</pub-id></citation></ref>
<ref id="B3">
<label>3.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>James</surname> <given-names>MT</given-names></name> <name><surname>Bhatt</surname> <given-names>M</given-names></name> <name><surname>Pannu</surname> <given-names>N</given-names></name> <name><surname>Tonelli</surname> <given-names>M</given-names></name></person-group>. <article-title>Long-term outcomes of acute kidney injury and strategies for improved care</article-title>. <source>Nat Rev Nephrol.</source> (<year>2020</year>) <volume>16</volume>:<fpage>193</fpage>&#x02013;<lpage>205</lpage>. <pub-id pub-id-type="doi">10.1038/s41581-019-0247-z</pub-id><pub-id pub-id-type="pmid">32051567</pub-id></citation></ref>
<ref id="B4">
<label>4.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Heung</surname> <given-names>M</given-names></name> <name><surname>Steffick</surname> <given-names>DE</given-names></name> <name><surname>Zivin</surname> <given-names>K</given-names></name> <name><surname>Gillespie</surname> <given-names>BW</given-names></name> <name><surname>Banerjee</surname> <given-names>T</given-names></name> <name><surname>Hsu</surname> <given-names>CY</given-names></name> <etal/></person-group>. <article-title>Acute kidney injury recovery pattern and subsequent risk of CKD: an analysis of veterans health administration data</article-title>. <source>Am J Kid Dis Offic J Nat Kidney Found.</source> (<year>2016</year>) <volume>67</volume>:<fpage>742</fpage>&#x02013;<lpage>52</lpage>. <pub-id pub-id-type="doi">10.1053/j.ajkd.2015.10.019</pub-id><pub-id pub-id-type="pmid">26690912</pub-id></citation></ref>
<ref id="B5">
<label>5.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Panitchote</surname> <given-names>A</given-names></name> <name><surname>Mehkri</surname> <given-names>O</given-names></name> <name><surname>Hastings</surname> <given-names>A</given-names></name> <name><surname>Hanane</surname> <given-names>T</given-names></name> <name><surname>Demirjian</surname> <given-names>S</given-names></name> <name><surname>Torbic</surname> <given-names>H</given-names></name> <etal/></person-group>. <article-title>Clinical predictors of renal non-recovery in acute respiratory distress syndrome</article-title>. <source>BMC Nephrol.</source> (<year>2019</year>) <volume>20</volume>:<fpage>1</fpage>&#x02013;<lpage>10</lpage>. <pub-id pub-id-type="doi">10.1186/s12882-019-1439-2</pub-id><pub-id pub-id-type="pmid">31362716</pub-id></citation></ref>
<ref id="B6">
<label>6.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Siew</surname> <given-names>ED</given-names></name> <name><surname>Abdel-Kader</surname> <given-names>K</given-names></name> <name><surname>Perkins</surname> <given-names>AM</given-names></name> <name><surname>Greevy</surname> <given-names>RA</given-names> <suffix>Jr</suffix></name> <name><surname>Parr</surname> <given-names>SK</given-names></name> <name><surname>Horner</surname> <given-names>J</given-names></name> <etal/></person-group>. <article-title>Timing of Recovery From Moderate to Severe AKI and the Risk for Future Loss of Kidney Function</article-title>. <source>Am J Kid Dis Offic J Nat Kidney Found.</source> (<year>2020</year>) <volume>75</volume>:<fpage>204</fpage>&#x02013;<lpage>13</lpage>. <pub-id pub-id-type="doi">10.1053/j.ajkd.2019.05.031</pub-id><pub-id pub-id-type="pmid">31537394</pub-id></citation></ref>
<ref id="B7">
<label>7.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Pannu</surname> <given-names>N</given-names></name> <name><surname>James</surname> <given-names>M</given-names></name> <name><surname>Hemmelgarn</surname> <given-names>B</given-names></name> <name><surname>Klarenbach</surname> <given-names>S</given-names></name></person-group>. <article-title>Association between AKI, recovery of renal function, and long-term outcomes after hospital discharge</article-title>. <source>Clin J Am Soc Nephrol.</source> (<year>2013</year>) <volume>8</volume>:<fpage>194</fpage>&#x02013;<lpage>202</lpage>. <pub-id pub-id-type="doi">10.2215/CJN.06480612</pub-id><pub-id pub-id-type="pmid">23124779</pub-id></citation></ref>
<ref id="B8">
<label>8.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Liang</surname> <given-names>KV</given-names></name> <name><surname>Sileanu</surname> <given-names>FE</given-names></name> <name><surname>Clermont</surname> <given-names>G</given-names></name> <name><surname>Murugan</surname> <given-names>R</given-names></name> <name><surname>Pike</surname> <given-names>F</given-names></name> <name><surname>Palevsky</surname> <given-names>PM</given-names></name> <etal/></person-group>. <article-title>Modality of RRT and recovery of kidney function after AKI in patients surviving to hospital discharge</article-title>. <source>Clinic J Am Soc Nephrol.</source> (<year>2016</year>) <volume>11</volume>:<fpage>30</fpage>&#x02013;<lpage>8</lpage>. <pub-id pub-id-type="doi">10.2215/CJN.01290215</pub-id><pub-id pub-id-type="pmid">26681135</pub-id></citation></ref>
<ref id="B9">
<label>9.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Macedo</surname> <given-names>E</given-names></name> <name><surname>Mehta</surname> <given-names>RL</given-names></name></person-group>. <article-title>Targeting recovery from acute kidney injury: incidence and prevalence of recovery</article-title>. <source>Nephron Clinic Pract.</source> (<year>2014</year>) <volume>127</volume>:<fpage>4</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1159/000363704</pub-id><pub-id pub-id-type="pmid">25343812</pub-id></citation></ref>
<ref id="B10">
<label>10.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Sawhney</surname> <given-names>S</given-names></name> <name><surname>Marks</surname> <given-names>A</given-names></name> <name><surname>Fluck</surname> <given-names>N</given-names></name> <name><surname>Levin</surname> <given-names>A</given-names></name> <name><surname>Prescott</surname> <given-names>G</given-names></name> <name><surname>Black</surname> <given-names>C</given-names></name></person-group>. <article-title>Intermediate and long-term outcomes of survivors of acute kidney injury episodes: a large population-based cohort study</article-title>. <source>Am J Kid Dis Offic J Nat Kidney Found.</source> (<year>2017</year>) <volume>69</volume>:<fpage>18</fpage>&#x02013;<lpage>28</lpage>. <pub-id pub-id-type="doi">10.1053/j.ajkd.2016.05.018</pub-id><pub-id pub-id-type="pmid">27555107</pub-id></citation></ref>
<ref id="B11">
<label>11.</label>
<citation citation-type="web"><person-group person-group-type="author"><name><surname>Itenov</surname> <given-names>TS</given-names></name> <name><surname>Berthelsen</surname> <given-names>RE</given-names></name> <name><surname>Jensen</surname> <given-names>JU</given-names></name> <name><surname>Gerds</surname> <given-names>TA</given-names></name> <name><surname>Pedersen</surname> <given-names>LM</given-names></name> <name><surname>Strange</surname> <given-names>D</given-names></name> <etal/></person-group>. <article-title>Predicting recovery from acute kidney injury in critically ill patients: development and validation of a prediction model</article-title>. <source>Critic Care Resuscit.</source> (<year>2018</year>) <volume>20</volume>:<fpage>54</fpage>. Available online at: <ext-link ext-link-type="uri" xlink:href="https://ccr.cicm.org.au/journal-search?searchtype=basic&#x00026;querytext=Predicting+recovery+from+acute+kidney+injury+in+critically+ill+patients%3a+development+and+validation+of+a+prediction+model">https://ccr.cicm.org.au/journal-search?searchtype=basic&#x00026;querytext=Predicting&#x0002B;recovery&#x0002B;from&#x0002B;acute&#x0002B;kidney&#x0002B;injury&#x0002B;in&#x0002B;critically&#x0002B;ill&#x0002B;patients%3a&#x0002B;development&#x0002B;and&#x0002B;validation&#x0002B;of&#x0002B;a&#x0002B;prediction&#x0002B;model</ext-link><pub-id pub-id-type="pmid">29458322</pub-id></citation></ref>
<ref id="B12">
<label>12.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>James</surname> <given-names>MT</given-names></name> <name><surname>Pannu</surname> <given-names>N</given-names></name> <name><surname>Hemmelgarn</surname> <given-names>BR</given-names></name> <name><surname>Austin</surname> <given-names>PC</given-names></name> <name><surname>Tan</surname> <given-names>Z</given-names></name> <name><surname>McArthur</surname> <given-names>E</given-names></name> <etal/></person-group>. <article-title>Derivation and external validation of prediction models for advanced chronic kidney disease following acute kidney injury</article-title>. <source>JAMA.</source> (<year>2017</year>) <volume>318</volume>:<fpage>1787</fpage>&#x02013;<lpage>97</lpage>. <pub-id pub-id-type="doi">10.1001/jama.2017.16326</pub-id><pub-id pub-id-type="pmid">29136443</pub-id></citation></ref>
<ref id="B13">
<label>13.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lee</surname> <given-names>BJ</given-names></name> <name><surname>Hsu</surname> <given-names>CY</given-names></name> <name><surname>Parikh</surname> <given-names>R</given-names></name> <name><surname>McCulloch</surname> <given-names>CE</given-names></name> <name><surname>Tan</surname> <given-names>TC</given-names></name> <name><surname>Liu</surname> <given-names>KD</given-names></name> <etal/></person-group>. <article-title>Predicting Renal Recovery After Dialysis-Requiring Acute Kidney Injury</article-title>. <source>Kidney Int Rep.</source> (<year>2019</year>) <volume>4</volume>:<fpage>571</fpage>&#x02013;<lpage>81</lpage>. <pub-id pub-id-type="doi">10.1016/j.ekir.2019.01.015</pub-id><pub-id pub-id-type="pmid">30993232</pub-id></citation></ref>
<ref id="B14">
<label>14.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Xiao</surname> <given-names>YQ</given-names></name> <name><surname>Cheng</surname> <given-names>W</given-names></name> <name><surname>Wu</surname> <given-names>X</given-names></name> <name><surname>Yan</surname> <given-names>P</given-names></name> <name><surname>Feng</surname> <given-names>LX</given-names></name> <name><surname>Zhang</surname> <given-names>NY</given-names></name> <etal/></person-group>. <article-title>Novel risk models to predict acute kidney disease and its outcomes in a Chinese hospitalized population with acute kidney injury</article-title>. <source>Sci Rep.</source> (<year>2020</year>) <volume>10</volume>:<fpage>1</fpage>&#x02013;<lpage>11</lpage>. <pub-id pub-id-type="doi">10.1038/s41598-020-72651-x</pub-id><pub-id pub-id-type="pmid">32973230</pub-id></citation></ref>
<ref id="B15">
<label>15.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hsu</surname> <given-names>CN</given-names></name> <name><surname>Liu</surname> <given-names>CL</given-names></name> <name><surname>Tain</surname> <given-names>YL</given-names></name> <name><surname>Kuo</surname> <given-names>CY</given-names></name> <name><surname>Lin</surname> <given-names>YC</given-names></name></person-group>. <article-title>Machine learning model for risk prediction of community-acquired acute kidney injury hospitalization from electronic health records: development and validation study</article-title>. <source>J Med Internet Res.</source> (<year>2020</year>) <volume>22</volume>:<fpage>e16903</fpage>. <pub-id pub-id-type="doi">10.2196/16903</pub-id><pub-id pub-id-type="pmid">32749223</pub-id></citation></ref>
<ref id="B16">
<label>16.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ogbadu</surname> <given-names>J</given-names></name> <name><surname>Singh</surname> <given-names>G</given-names></name> <name><surname>Aggarwal</surname> <given-names>D</given-names></name></person-group>. <article-title>Factors affecting the transition of acute kidney injury to chronic kidney disease: Potential mechanisms and future perspectives</article-title>. <source>Eur J Pharmacol.</source> (<year>2019</year>) <volume>865</volume>:<fpage>172711</fpage>. <pub-id pub-id-type="doi">10.1016/j.ejphar.2019.172711</pub-id><pub-id pub-id-type="pmid">31589870</pub-id></citation></ref>
<ref id="B17">
<label>17.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Toma&#x00161;ev</surname> <given-names>N</given-names></name> <name><surname>Glorot</surname> <given-names>X</given-names></name> <name><surname>Rae</surname> <given-names>JW</given-names></name> <name><surname>Zielinski</surname> <given-names>M</given-names></name> <name><surname>Askham</surname> <given-names>H</given-names></name> <name><surname>Saraiva</surname> <given-names>A</given-names></name> <etal/></person-group>. <article-title>A clinically applicable approach to continuous prediction of future acute kidney injury</article-title>. <source>Nature.</source> (<year>2019</year>) <volume>572</volume>:<fpage>116</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1038/s41586-019-1390-1</pub-id><pub-id pub-id-type="pmid">31367026</pub-id></citation></ref>
<ref id="B18">
<label>18.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Shao</surname> <given-names>SC</given-names></name> <name><surname>Chan</surname> <given-names>YY</given-names></name> <name><surname>Kao Yang</surname> <given-names>YH</given-names></name></person-group>. <article-title>The Chang Gung research database-a multi-institutional electronic medical records database for real-world epidemiological studies in Taiwan</article-title>. <source>Pharmacoepidemiol Drug Saf.</source> (<year>2019</year>) <volume>28</volume>:<fpage>593</fpage>&#x02013;<lpage>600</lpage>. <pub-id pub-id-type="doi">10.1002/pds.4713</pub-id><pub-id pub-id-type="pmid">30648314</pub-id></citation></ref>
<ref id="B19">
<label>19.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Collins</surname> <given-names>GS</given-names></name> <name><surname>Reitsma</surname> <given-names>JB</given-names></name> <name><surname>Altman</surname> <given-names>DG</given-names></name> <name><surname>Moons</surname> <given-names>KG</given-names></name></person-group>. <article-title>Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement</article-title>. <source>The TRIPOD Group Circul.</source> (<year>2015</year>) <volume>131</volume>:<fpage>211</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.1161/CIRCULATIONAHA.114.014508</pub-id><pub-id pub-id-type="pmid">25627261</pub-id></citation></ref>
<ref id="B20">
<label>20.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Stevens</surname> <given-names>LM</given-names></name> <name><surname>Mortazavi</surname> <given-names>BJ</given-names></name> <name><surname>Deo</surname> <given-names>RC</given-names></name> <name><surname>Curtis</surname> <given-names>L</given-names></name> <name><surname>Kao</surname> <given-names>DP</given-names></name></person-group>. <article-title>Recommendations for reporting machine learning analyses in clinical research</article-title>. <source>Circul Cardiovascul Qual Outcomes.</source> (<year>2020</year>) <volume>13</volume>:<fpage>e006556</fpage>. <pub-id pub-id-type="doi">10.1161/CIRCOUTCOMES.120.006556</pub-id><pub-id pub-id-type="pmid">33079589</pub-id></citation></ref>
<ref id="B21">
<label>21.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Guthrie</surname> <given-names>G</given-names></name> <name><surname>Guthrie</surname> <given-names>B</given-names></name> <name><surname>Walker</surname> <given-names>H</given-names></name> <name><surname>James</surname> <given-names>MT</given-names></name> <name><surname>Selby</surname> <given-names>NM</given-names></name> <name><surname>Tonelli</surname> <given-names>M</given-names></name> <etal/></person-group>. <article-title>Developing an AKI consensus definition for database research: findings from a scoping review and expert opinion using a delphi process</article-title>. <source>Am J Kid Dis.</source> (<year>2021</year>) <volume>5</volume>:<fpage>19</fpage>. <pub-id pub-id-type="doi">10.1053/j.ajkd.2021.05.019</pub-id><pub-id pub-id-type="pmid">34298142</pub-id></citation></ref>
<ref id="B22">
<label>22.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Youden</surname> <given-names>WJ</given-names></name></person-group>. <article-title>Index for rating diagnostic tests</article-title>. <source>Cancer</source>. (<year>1950</year>) <volume>3</volume>:<fpage>32</fpage>&#x02013;<lpage>35</lpage>.</citation>
</ref>
<ref id="B23">
<label>23.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Ribeiro</surname> <given-names>MT</given-names></name> <name><surname>Singh</surname> <given-names>S</given-names></name> <name><surname>Guestrin</surname> <given-names>C</given-names></name></person-group>. <article-title>Why should i trust you?</article-title> In: <source>Explaining the predictions of any classifier. Paper presented at: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining</source> (<year>2016</year>).</citation>
</ref>
<ref id="B24">
<label>24.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Lundberg</surname> <given-names>SM</given-names></name> <name><surname>Lee</surname> <given-names>SI</given-names></name></person-group>. <article-title>A unified approach to interpreting model predictions,</article-title> In: <source>Paper Presented at: Proceedings of the 31st International Conference on Neural Information Processing Systems</source> (<year>2017</year>).</citation>
</ref>
<ref id="B25">
<label>25.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Kellum</surname> <given-names>JA</given-names></name> <name><surname>Sileanu</surname> <given-names>FE</given-names></name> <name><surname>Bihorac</surname> <given-names>A</given-names></name> <name><surname>Hoste</surname> <given-names>EAJ</given-names></name> <name><surname>Chawla</surname> <given-names>LS</given-names></name></person-group>. <article-title>Recovery after acute kidney injury</article-title>. <source>Am J Respirat Critic Care Med.</source> (<year>2017</year>) <volume>195</volume>:<fpage>784</fpage>&#x02013;<lpage>91</lpage>. <pub-id pub-id-type="doi">10.1164/rccm.201604-0799OC</pub-id><pub-id pub-id-type="pmid">27635668</pub-id></citation></ref>
<ref id="B26">
<label>26.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Siew</surname> <given-names>ED</given-names></name> <name><surname>Ikizler</surname> <given-names>TA</given-names></name> <name><surname>Matheny</surname> <given-names>ME</given-names></name> <name><surname>Shi</surname> <given-names>Y</given-names></name> <name><surname>Schildcrout</surname> <given-names>JS</given-names></name> <name><surname>Danciu</surname> <given-names>I</given-names></name> <etal/></person-group>. <article-title>Estimating baseline kidney function in hospitalized patients with impaired kidney function</article-title>. <source>Clin J Am Soc Nephrol.</source> (<year>2012</year>) <volume>7</volume>:<fpage>712</fpage>&#x02013;<lpage>9</lpage>. <pub-id pub-id-type="doi">10.2215/CJN.10821011</pub-id><pub-id pub-id-type="pmid">28071065</pub-id></citation></ref>
<ref id="B27">
<label>27.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Siew</surname> <given-names>ED</given-names></name> <name><surname>Parr</surname> <given-names>SK</given-names></name> <name><surname>Wild</surname> <given-names>MG</given-names></name> <name><surname>Levea</surname> <given-names>SL</given-names></name> <name><surname>Mehta</surname> <given-names>KG</given-names></name> <name><surname>Umeukeje</surname> <given-names>EM</given-names></name> <etal/></person-group>. <article-title>Kidney disease awareness and knowledge among survivors ofacute kidney injury</article-title>. <source>Am J Nephrol.</source> (<year>2019</year>) <volume>49</volume>:<fpage>449</fpage>&#x02013;<lpage>59</lpage>. <pub-id pub-id-type="doi">10.1159/000499862</pub-id><pub-id pub-id-type="pmid">30995659</pub-id></citation></ref>
<ref id="B28">
<label>28.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Silver</surname> <given-names>SA</given-names></name> <name><surname>Adhikari</surname> <given-names>NK</given-names></name> <name><surname>Bell</surname> <given-names>CM</given-names></name> <name><surname>Chan</surname> <given-names>CT</given-names></name> <name><surname>Harel</surname> <given-names>Z</given-names></name> <name><surname>Kitchlu</surname> <given-names>A</given-names></name> <etal/></person-group>. <article-title>Nephrologist follow-up vs. usual care after an acute kidney injury hospitalization (FUSION): a randomized controlled trial</article-title>. <source>Clinic J Am Soc Nephrol.</source> (<year>2021</year>) <volume>16</volume>:<fpage>1005</fpage>&#x02013;<lpage>14</lpage>. <pub-id pub-id-type="doi">10.2215/CJN.17331120</pub-id><pub-id pub-id-type="pmid">34021031</pub-id></citation></ref>
<ref id="B29">
<label>29.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Hansrivijit</surname> <given-names>P</given-names></name> <name><surname>Yarlagadda</surname> <given-names>K</given-names></name> <name><surname>Puthenpura</surname> <given-names>MM</given-names></name> <name><surname>Ghahramani</surname> <given-names>N</given-names></name> <name><surname>Thongprayoon</surname> <given-names>C</given-names></name> <name><surname>Vaitla</surname> <given-names>P</given-names></name> <etal/></person-group>. <article-title>A meta-analysis of clinical predictors for renal recovery and overall mortality in acute kidney injury requiring continuous renal replacement therapy</article-title>. <source>J Crit Care.</source> (<year>2020</year>) <volume>60</volume>:<fpage>13</fpage>&#x02013;<lpage>22</lpage>. <pub-id pub-id-type="doi">10.1016/j.jcrc.2020.07.012</pub-id><pub-id pub-id-type="pmid">32731101</pub-id></citation></ref>
<ref id="B30">
<label>30.</label>
<citation citation-type="journal"><person-group person-group-type="author"><collab>Kidney Disease: Improving Global Outcomes (KDIGO)</collab></person-group>. <article-title>Acute Kidney Injury Work Group KDIGO clinical practice guidelines for acute kidney injury</article-title>. <source>Kidney Int. Suppl</source>. (<year>2012</year>) <volume>2</volume>:<fpage>1</fpage>&#x02013;<lpage>138</lpage>. <pub-id pub-id-type="doi">10.1038/kisup.2012.1</pub-id></citation>
</ref>
<ref id="B31">
<label>31.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Abu-Alfa</surname> <given-names>AK</given-names></name> <name><surname>Younes</surname> <given-names>A</given-names></name></person-group>. <article-title>Tumor lysis syndrome and acute kidney injury: evaluation, prevention, and management</article-title>. <source>Am J Kidney Dis.</source> (<year>2010</year>) <volume>55</volume>:<fpage>S1</fpage>&#x02013;<lpage>S13</lpage>. <pub-id pub-id-type="doi">10.1053/j.ajkd.2009.10.056</pub-id><pub-id pub-id-type="pmid">20420966</pub-id></citation></ref>
<ref id="B32">
<label>32.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Porta</surname> <given-names>C</given-names></name> <name><surname>Cosmai</surname> <given-names>L</given-names></name> <name><surname>Gallieni</surname> <given-names>M</given-names></name> <name><surname>Pedrazzoli</surname> <given-names>P</given-names></name> <name><surname>Malberti</surname> <given-names>F</given-names></name></person-group>. <article-title>Renal effects of targeted anticancer therapies</article-title>. <source>Nat Rev Nephrol.</source> (<year>2015</year>) <volume>11</volume>:<fpage>354</fpage>&#x02013;<lpage>70</lpage>. <pub-id pub-id-type="doi">10.1038/nrneph.2015.15</pub-id><pub-id pub-id-type="pmid">27521117</pub-id></citation></ref>
<ref id="B33">
<label>33.</label>
<citation citation-type="journal"><person-group person-group-type="author"><name><surname>Weller</surname> <given-names>S</given-names></name> <name><surname>Varrier</surname> <given-names>M</given-names></name> <name><surname>Ostermann</surname> <given-names>M</given-names></name></person-group>. <article-title>Lymphocyte function in human acute kidney injury</article-title>. <source>Nephron.</source> (<year>2017</year>) <volume>137</volume>:<fpage>287</fpage>&#x02013;<lpage>93</lpage>. <pub-id pub-id-type="doi">10.1159/000478538</pub-id><pub-id pub-id-type="pmid">28662513</pub-id></citation></ref>
</ref-list>
</back>
</article>