Chronic kidney disease (CKD) is a major complication arising from diabetes, and around 25%–40% of patients with diabetes develop diabetic kidney disease (1). Diabetes is not only one of the most common diseases but also the leading cause of kidney failure. In Singapore, diabetes accounts for two-thirds of cases of end-stage kidney failure (ESKF) requiring dialysis, with the prevalence being one of the highest in the world according to the United States Renal Data System (USRDS)’s data (https://adr.usrds.org/2021/end-stage-renal-disease/11-international-comparisons). Diabetes leads to kidney damage via two main pathways: chronic hyperglycemia and the activation of the renin–angiotensin system. These processes ultimately induce glomerular sclerosis, albuminuria, and kidney impairment (2). CKD is projected to be the fifth leading cause of death globally by 2040 (3). Because it is often asymptomatic in the early stages, CKD is usually diagnosed late. The treatment for patients with ESKF requires substantial resources and is still a great challenge to a healthcare system that is under resource limits and budget control. The patients themselves also suffer from a significant loss of quality of life and from the financial burdens affecting them as a result of their condition.

It is critical to identify patients who are at high risk of kidney function deterioration to avert or delay their progression to ESKF. These patients could be treated in the early stages and be engaged early on in their long-term kidney care plans. The Nephrology Evaluation, Management and Optimization (NEMO) program was introduced to optimize renal protection treatment earlier within the primary care setting. Compared with diabetic patients not enrolled in NEMO, the patients in the NEMO program had a 28% lower rate of kidney function deterioration (https://www.healthhub.sg/a-z/medical-and-care-facilities/nemo-programme-for-diabetic-kidney-patients). However, the NEMO program focused only on patients from one polyclinic cluster in Singapore and addressed only one treatment parameter. Consequently, the Holistic Approach in Lowering and Tracking Chronic Kidney Disease (HALT-CKD) program, which is comprised of nephrologists from all public hospitals in Singapore, was launched in 2017 as an enhanced and extended version of the NEMO program. The HALT-CKD national program targets and tracks a much broader cohort, that is, all CKD patients from stage 1 to stage 4 in Singapore. It is also carried out at all polyclinics across Singapore and aims to identify and control evidence-based risk factors of CKD to delay CKD progression.

A prediction method of CKD progression has been studied in some western countries’ populations to assist in the early intervention of patients. Tangri et al. (4) developed a risk model of CKD progression to ESKF in the Canadian population and found a list of important indicators—age, sex, estimated glomerular filtration rate (eGFR), and levels of urine albumin, serum calcium, serum phosphate, serum bicarbonate, and serum albumin. Later, this model was evaluated and recalibrated for multinational cohorts (5). The method of Tangri et al. was also applied to predict the probability and the timing of kidney failure that requires kidney replacement therapy (6). Ramspeck et al. (7) selected 11 prediction models on kidney failure and validated them on two large cohorts in Europe. The model’s performance on the validation cohorts was considerably worse than on the original cohorts due to the narrower patient mix of the validation cohorts. This result indicates the necessity of retraining and even redesigning the CKD prediction model when applying it to populations with different ethnicities.

Low et al. (8) followed 1,582 patients with type 2 diabetes mellitus, from 2002 to 2014, and developed a logistic regression model for CKD progression specifically for diabetic patients in one Singapore hospital. In recent years, machine-learning algorithms, such as random forest and XGBoost, were also used to create the risk models of CKD progression, which showed a potential for improvement from statistical methods (9, 10).

However, there are some limitations in the previous studies. First, the study population did not cover the different ethnic groups in Singapore and was not representative of all diabetic patients in the country. Second, there was no clinical validation to confirm the viability of the models and to demonstrate their clinical usefulness in practice. Some studies did not even have validation datasets to check the stability of the model’s performance.

In the current study, we developed a predictive model for CKD progression in 5 years for diabetic patients in Singapore. Our study used the nationwide medical data of patients in Singapore by utilizing the two national systems: the National Diabetes Database (NDD) and the Business Research Analytics Insights Network (BRAIN). Our study result has been validated technically and clinically by nephrologists from public hospitals, hence the endorsement from the HALT-CKD program’s committee to incorporate it into its action plan.

In the current study, we developed a model for CKD progression in diabetic patients. Our study cohort contained 19,050 adult patients diagnosed with diabetes and CKD stages 1 and 2 before 2013; 37.3% of them deteriorated into CKD stage 3 or above in 5 years. Table 2 shows the descriptive summary of the study cohort. The training, test, and validation datasets all showed similar characteristics as shown in Table 2 .

Following our feature selection process, the original 177 variables were narrowed down to four categories of variables and used for training our CKD model, as elaborated below. The sociodemographic characteristics included age, gender, and race. The diagnosis variable included diabetes duration, which was the number of days from the earliest diabetes diagnosis date to the date of prediction, i.e., 1 January 2013. The variables derived from the laboratory test history included measurements of hemoglobin A_1C, albumin urine, glucose, hemoglobin, the urine albumin-to-creatinine ratio, albumin serum, eGFR, alanine aminotransferase, triglycerides, high-density lipoprotein, low-density lipoprotein, and total cholesterol. The glomerular filtration rate (GFR) is an important metric that reflects kidney function, but measurement requires a complicated and lengthy procedure. The eGFR is an approximate to glomerular filtration rate. It is calculated using the CKD-EPI equation based on serum creatinine, age, gender, and ethnicity (non-Black www.kidney.org/content/ckd-epi-creatinine-euqation-2009). The statistics of these laboratory tests are shown in Table 2 . The “prior” laboratory readings in Table 2 refer to the test result closest to but at least 90 days before the latest laboratory reading before 1 January 2013. This gives an indication of the magnitude of the change in the laboratory results and provides a sense of the progression of the patients’ clinical conditions and parameters.

The medication data were standardized into the groups from the Anatomical Therapeutic Chemical Classification (ATC) of which the first level contains 14 main anatomical and pharmacological groups (15). For example, an ATC code with the first level “A” tells that the drug is related to the alimentary tract or metabolism, while the ATC code with the first level “C” tells that the drug acts on the cardiovascular system.

Figure 3 displays the important predictive factors for the CKD progression in our model. Consistent with the findings in previous studies (4, 16, 17), the leading factors include eGFR and albumin urine concentration. Interestingly, we found that the inclusion of medication history, specifically drugs for the cardiovascular system and metabolism, and other laboratory tests such as hemoglobin levels improved the model’s performance in predicting the 5-year risk of CKD progression for diabetic patients.

Our developed CKD model outperformed the current clinical practice and the mainstream statistical models for predicting CKD progression. Clinicians use the eGFR to diagnose CKD in current practice. We developed a Cox regression model using only an eGFR to mimic the clinicians’ assessment. Additionally, we developed another Cox regression model using the features in Tangri’s paper (4), which was widely recognized and had a study design similar to ours. Table 3 summarizes the performance of our models and other major models. Our model had the best performance in terms of area under the receiver operating characteristic (ROC) curve (AUC) value. The 95% confidence interval of our model’s AUC is 0.87 to 0.89, which is significantly higher than other statistical models.

In addition, we tested the stability of our model’s performance on a new cohort by shifting 6 months. The new cohort consisted of the patients using the same exclusion criteria as in Figure 2 but shifting the ending date to 30 June 2013 instead of 31 December 2012. The new cohort did not overlap with the study cohort discussed above. Our model predicted their risk of CKD progression in the next 5 years until 1 July 2018. As Table 3 shows, our model’s performance was stable, indicating its potential to be generalized to new patients in different periods.

We have further validated our developed model clinically by comparison with clinicians tasked with predicting CKD progression clinically. We generated a new 2013 diabetes cohort of 1,000 patients as described in Methods and provided their features used by our model to 20 clinicians across all three polyclinics groups in Singapore (i.e., NHGP, NUP, and SHP). These 1,000 patients were not included in the original 2012 cohort that was used to develop the model or in the 6-month shift-test data. We asked the clinicians to assess the patients’ risk of CKD progression in the next 5 years and then compared the performance of their assessment and our model.

Figure 4A displays the receiver operating characteristic (ROC) of our model, in which the red dots represent the combinations of 1 – specificity and the sensitivity of the clinicians’ prediction. The area under the ROC curve (AUC) of the clinical validation data was still 0.88, which demonstrated, again, that our model could be generalized to new patients. Figure 4B shows the precision–recall curve of our model, in which the red dots represent the combinations of sensitivity and the precision of the clinicians’ predictions. The predictions by clinicians varied across seniority levels. Some clinicians did better than others in their assessments. However, our model outperforms all clinician subgroups because all the dots are under the curves in both Figures 4A, B . For example, our model was able to achieve a 0.74 sensitivity and a 0.81 precision in the clinical validation and it could also flexibly achieve the other balance along the curve in Figure 4B . Compared with clinicians of different seniority levels, our model predicted more accurately.

CKD is a major complication and concern for diabetic patients, and two-thirds of new kidney failure cases are due to diabetes in Singapore. Identifying high-risk patients and intervening early will help prevent kidney failure, improve patient outcomes, and utilize hospital resources more cost-effectively. Our research contributes to identifying high-risk patients in the early stages of CKD. It is also different from previous studies in the following ways.

First, our project was the first national-level project on kidney diseases in diabetic patients. The study used data from the national cohort that covered the entire diabetic population in Singapore. The diabetic patients’ data, which were previously scattered among diabetic registries owned by different hospitals and the MOH, were collected and consolidated by the NDD. It contains the most complete datasets of diabetic patients in Singapore and is kept up to date with new data via the BRAIN platform. The national cohort represents the diverse population in Singapore, which includes different ethnicity groups from south and east Asia. As Ta et al. (11) introduced, the BRAIN platform runs on the existing infrastructure and draws the most current data in a timely manner from the source system. It has six core elements: (i) secure access channels, which enable the BRAIN platform to connect to multiple data sources; (ii) a national health identification service, which enables the BRAIN to uniquely identify people and their records received from different health domains; (iii) an enterprise terminology service, which enables the BRAIN platform to harmonize data across multiple source systems; (iv) a de-identification service, which supports the de-identification of patient data before presenting them to researchers or prediction tools, and the re-identification of patient data when needed; (v) user groups, which include data analysts, policymakers, and stakeholders; and (vi) tools for model and dashboard developments. Tools such as Python, R, Stata, and Tableau are available on the BRAIN platform. The BRAIN platform and NDD database are readily available to stimulate more research on diabetes in the future.

Second, our research identified new indicative factors that improve the prediction accuracy. In addition to the eGFR and the concentration of albumin urine, which have been proven to be predictive in previously developed models, our model found that medication history (e.g., drugs for the cardiovascular system and metabolism) and other laboratory tests (e.g., hemoglobin) may also play an important role in predicting CKD progression in diabetic patients. Some features that we used have correlations with each other, for example, urine albumin and the urine albumin-to-creatinine ratio. We kept both features because they can provide added value to predict CKD progression. Removing one will decrease the model’s performance on the validation set.

Third, our research developed a machine-learning algorithm that outperforms the current clinical practice and the mainstream statistical models in predicting CKD progression, as indicated in the results above. The technical validation of a 6-month shifted cohort also confirmed that our developed model is stable by showing the same good performance (with an AUC of 0.88) as the original base cohort. No calibration was needed when applying to different validation data, as our model was well trained using a representative national cohort in Singapore.

Fourth, our model performed better than clinicians in predicting CKD progression. More importantly, our model also reduced the variation of prediction among clinicians and provided a higher and more consistent accuracy.

Fifth, our model was the first clinical model endorsed by the MOH and the HALT-CKD program in Singapore to be deployed nationwide by incorporating it into the HALT-CKD program, which was established to prevent and slow down the deterioration of CKD in Singapore, with many nephrologists from public hospitals in its committee. Figure 5 shows how our model supports the HALT-CKD program to achieve more targeted early intervention. The cohorts of diabetic patients and their data were consolidated in the NDD. The ETL jobs have been scheduled to automatically update the NDD data daily, transform the data into the model inputs, and trigger the CKD model to generate risk scores for CKD1/2 patients once a month. The predicted score generated by the model can be flexibly banded into low, high, and very high risk to allow for the allocation of healthcare resources accordingly. The score is displayed on a diabetic patient’s dashboard together with other relevant data.

The two dashboards, as demonstrated in Supplementary Figures 2 , 3 , were derived and integrated into the electronic health system. The Diabetes National Dashboard is a national-level dashboard that provides a snapshot of the diabetic profile. It provides cohort insights and supports decision-making for health policies and budget planning. The other dashboard is an individual-level dashboard, which incorporates the CKD model and consolidates key information for each patient, for example, laboratory tests and appointment information. It not only saves clinicians, who normally suffer from heavy clinical workloads, from carrying out the risk assessment of CKD progression but also provides improved accuracy when identifying the patients at high risk who may otherwise be missed given the time constraints of the consultation.

When patients come to visit clinicians, especially those in primary care who are the first line of care, the CKD stage 1/2 patients with a high risk of CKD progression will be flagged in the dashboard. The clinicians can view and print the dashboard results from the electronic health system, use it as a tool for patient coaching, and implement more targeted early intervention for them. For example, in the HALT-CKD program, there are two phases of intervention. In phase 1, patients with a high and very high risk will have more intensive drug therapy intervention, such as more frequent drug titration. In phase 2, which is more resource intensive, patients with a very high risk will be selected for lifestyle modification. Considering the 711,800 diabetic patients aged from 20 to 79 years old in Singapore in 2021 ² , approximately 474,500 diabetic patients, i.e., two-thirds of diabetic patients, will end up with kidney failure. By integrating the prediction model into the clinical workflow, these patients could benefit from early CKD intervention. The enhanced performance of the model enables the identification of more patients at a high risk of kidney deterioration. In general, the individual-level dashboard can fit into the current clinical workflow and enhance the clinicians’ ability to care for patients.

However, our study has a number of limitations, thus leaving room for future study. First, patients’ data from private health institutes are not available although they only constitute a small portion of the healthcare data of Singapore. This affects data completeness and might have influenced the model’s performance. Second, the impact of lifestyle factors, such as physical exercise and diet, has not been considered in the model. The model’s performance may be further enhanced by involving lifestyle factors. Third, our study is targeted at diabetic patients in Singapore, who mainly consist of Chinese, Malay, and Indian people. The model’s performance may decline, and the model should be retrained if applied to populations with different ethnicities.

In conclusion, our risk model of CKD progression for diabetic patients demonstrated its strength in prediction performance. It outperformed other predictive models and clinicians’ judgements in clinical validation. It has been endorsed by the MOH and the HALT-CKD to be deployed nationwide in Singapore to support clinicians in the assessment and medical treatment of patients in the early stages of CKD. Feedback will be collected from clinicians and the program committee to improve the predictive model further in the future. This project provided a successful example to demonstrate how an artificial intelligence (AI)-based model can be adopted to support clinical decision-making nationwide.

The original contributions presented in the study are included in the article/ Supplementary Material . Further inquiries can be directed to the corresponding author.

The studies involving humans data were approved by the National Health Group Singapore. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

ZF and WT contributed to the study design and main concept; ZF and ZW performed data analysis, developed the model, and drafted the paper. KC, MJ, and KP deployed the model. AY, GA, AL, CL, MF, and WC contributed to the data acquisition, designed the application of the model, and integrated the model into the national program. AL and CL revised the manuscript for important intellectual content. All authors read and approved the final draft before submission. WT is the guarantor of this work.