The primary data sources for our study are the de-identified Medicare inpatient and skilled nursing facility files of the Centers for Medicare and Medicaid Services (CMS) Limited Use Datasets (LDS). The dataset contains 6,567,071 Medicare beneficiary claims from 2012 to 2019, which comprise 5% of the total Medicare beneficiaries. CMS is a federal health care program in the United States that is mainly composed of residents who are 65 and older. From this data set, we identified 282 MDD patients who progressed to AD and a randomly selected equal number of patients with MDD who never progressed to AD between 2012 and 2019.

We used the International Classification of Diseases, Clinical Modification (ICD-CM) coding systems, including ICD-9-CM and ICD-10-CM, to identify depression and AD diagnoses. The ICD-9-CM diagnosis applies to data from 2012 to 2015, and the ICD-10-CM diagnosis applies to data from 2016 to 2019.

Together with the CMS LDS, we used the county-level Health Professional Shortage Areas (HPSA) classification to identify the shortage of health professionals in each patient's county and the 2013 county-level Economic Research Service Rural-Urban Continuum Codes (RUCCs) to classify whether an observation is from a designated rural or urban county (Hardeman et al., 2022; Tan et al., 2022). The Office of Management and Budget uses the RUCC to classify large metros with at least 1 million residents and small metro areas with <1 million residents as urban. Other non-metropolitan regions are classified as rural.

We operationalized MDD as clinical depression that includes all three severity levels of depression (mild, moderate, and severe) for single (F.32) and recurrent (F.33), including recurrent brief depressive episodes (Hasin et al., 2018). We consider 296.2X for ICD-9-CM and the F.32.X for ICD-10-CM (Association, 2015). According to the DSM-5, MDD is characterized by a minimum of five depressive mood symptoms presenting as feelings of sadness, emptiness, hopelessness, tremendous weight loss or daily bidirectional changes in appetite, insomnia or hypersomnia, agitation, fatigue, feelings of worthlessness, reduced cognitive or executive functions, and thoughts of suicide.

Using the ICD codes in the CMS LDS, we identified AD with ICD-9-CM code 331.0 and ICD-10-CM codes G30.0, G30.1, G30.8, and G30.9 (Sachdev et al., 2014). Based on the DSM-5, AD is a progressive neurodegenerative disease that causes the death of nerve cells in various parts of the brain, mainly in the frontotemporal region. This loss of function results in a decline of cognitive abilities such as memory and language.

We learned the underlying representation of the different levels of abstraction of the covariates and predictors using an efficient backward propagation (backpropagation) method for the Multi-Layer Perceptron (MLP) deep learning algorithm (Zhou et al., 2021). The backpropagation method uses the chain rule, enabling the MLP to systematically understand the underlying structure in reverse order, beginning from the output layer of the network (outcome variable) to the input layer (covariates and predictors). Thus, the MLP considers the complex underlying representation of the observational data and the contextual interaction among the covariates and predictors, often challenging to identify by standard regression models and incredibly difficult to calculate manually. The underlying abstraction of the data is captured as the weights connecting several internal layers of the predictor and covariate structures. A unique underlying weight structure adequately predicts the progression from depression to AD. Similarly, the control group, non-progressive depression (no progression from depression to AD), has an underlying weight structure, which is inherently different from the progression group.

We performed a grid search to identify optimal values for the MLP. Parameter values include the rectified linear unit (ReLU) activation function, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (lbfgs) solver, an alpha of 1e-8, tolerance level of 1e-3, learning rate initialization of 0.1, hidden layers of 6, max iteration of 1,200, and a random state of 4. ReLU is known for its exemplary performance in computing the predicted probabilities for the underlying weight structure (Hara and Saito, 2015). Also, the lbfgs is known for its efficiency on small to medium datasets, as observed in our training set. Finally, we trained the MLP model on 80% of the dataset and tested it with the remaining 20%.

The proposed fully connected structural representation of the observational data using the MLP architecture is presented in this section. With the predictors and covariates fed as inputs to the network structure, the hidden layer helps compute the weights using a specified activation function. The weights capture the underlying pattern within the data. The MLP then classifies the output as one of the outcome categories. Figure 2 depicts a simple three-layer structure of an MLP model.

MSM is a class of causal models on observational data, which accounts for time-varying confounders when examining the effect of a time-dependent exposure. Its use has been encouraged to eliminate bias in observational data (Athey, 2015; Prosperi et al., 2020).

Thus, we present the causal model as a directed acyclic graph (DAG). The DAG indicates the time-varying covariates, confounders, and the underlying connection with MDD diagnosis and the risk of progression to AD. In the DAG, X indicates the factors affecting the progression from MDD to AD, E denotes the exposure for having been diagnosed with MDD, U is the unknown factor(s) affecting the outcome, and Y is the outcome, which is the progression from MDD to AD. The DAG shows that the probability of being diagnosed with MDD at E(0) is conditioned on covariates and confounding factors X(0) in the data. E(1) and X(1) are conditioned on E(0) and X(0). The inverse of these conditional probabilities of receiving exposure E(1) given E(0) and X(0) are referred to as the inverse probability of treatment weights (IPTW; Austin, 2016). A priori, the IPTW for the MSM is calculated using the following equation:

where t indicates the time lag in years for T number of years with t = 0 as the baseline year and t−1 as the 1-year time lag, E_t indicates the MDD diagnosis at a particular time or time lag in years. X is a set of time-varying and non-time-varying confounders. The conditional probabilities for the numerator and denominator were computed using the predicted probabilities from a logistic regression model that controls for the non-time-varying and time-varying confounders. Finally, the IPTW computes a stabilized weight that controls for time-varying confounding variables. It provides better causal estimates on observational data. Figure 3 depicts a simple MSM.

Our integrated model combines the MSM and MLP models using an ensemble learning method called the Gradient Boosting Classier (GBC; Wade, 2020). This method uses an additive model to reflect the contribution of each model. Hyperparameter tuning was performed to control overfitting by introducing randomness to the model using a 60% subsample of the training set for each boosting iteration without replacement. Thus, the parameters for the GBC include random_state = 28, n_estimators = 100, learningrate = 0.01, max_depth = 3, and subsample = 0.6. Also, each model in the ensemble model used the same training data. The integrated prediction for progression from MDD to AD is computed using Equation 2 as follows:

where IPM_i is the integrated prediction model for the i-th observation, pANN_i indicates the MLP model prediction for the i-th observation, and pMSM_i is the MSM prediction for the i-th observation. Thus, we used the IPM to classify unseen observations likely to progress from MDD to AD. The IPM weighs the predictions from the pANN and pMSM models and effectively integrates them as features for the final predictive outcome. The predictive features from each model represent varying aspects of the underlying patterns of the data. More importantly, the MSM features capture marginal relationships between the variables, while the MLP features capture more complex, non-linear patterns from the data. Therefore, the IPM leverages the diverse feature representations to make the final predictions.

Further, we evaluated the model using the Area Under the Curve (AUC), accuracy, precision, recall, and the f1-score (Jiao, 2016; Carrington et al., 2022).

We performed sensitivity analysis (SA) as part of the safeguards for prediction modeling and explainable artificial intelligence models from observational data (Smith, 2021). Specifically, we demonstrated the robustness and precision of the IPM using external validation techniques (Peters and Bühlmann, 2016). Thus, we deliberately distorted the original dataset by randomly replacing 40% of the data with missing values across all the predictors. Further, we used the Markov Chain Monte Carlo multiple imputation technique to impute the missing values using the original data's predictive distribution (multivariate normal distribution; Rezvan and Lee, 2015). We then evaluated the IPM on the imputed dataset and compared findings to the original dataset.

To show the similarity between the characteristics of patients who remained MDD and progressed to AD, we conducted the chi-square test of independence on all the categorical variables (age, gender, race, rural-urban, HPSA, comorbidity, and follow-up visit). Further, we performed a two-sided independent sample t-test on all numerical variables (follow-up visit and count of claims for MDD). We conducted t-tests, chi-square tests, and logistic regression models for computing the IPTW stabilized weights (Equation 1) using the Statistical Analysis Software (SAS) version 9.4 on SAS Studio. All statistical tests were performed at a 95% confidence interval (CI). We also used Python 3.12.1 to develop the MSM with the Weighted Least Squares (WLS) estimation function of the statsmodels module version 0.15.0. The WLS function takes the IPTW as weights to fit the MSM and generates the predictive probabilities. Finally, we used the Scikit-learn version 1.1 Python machine learning module to implement the MLP model and the Gradient Boosting Classifier that performed the IPM. Note that 80% of the patients per group were used as the training set. There were 282 beneficiaries per group (remained MDD vs. progressed to AD), which equates to ~226 patients per group for training and ~56 patients for testing. The exact proportions were used across the IPM, MSM, and MLP.

Table 1 shows the descriptive characteristics of the samples used in developing our model. A total of 282 unique patients met the definition of the outcome variable by progressing from MDD to AD diagnosis. A randomly selected group of 282 patients who remained MDD as of 2019 was used as a control group. Patient demographics were broadly similar across groups par the statistical significance tests at 95% CI. Descriptively, there were more older patients who progressed to AD compared to those who remained MDDs. Both groups contained a higher percentage of female than male patients (remained MDD 59.22% and progressed to AD 65.96%). White was the most common ethnicity in both groups (remained MDD 85.46% and progressed to AD 86.88%) groups. Most patients lived in rural locations (remained MDD 82.62% and progressed to AD 72.34%), experiencing partial healthcare shortages (remained MDD 56.03% and progressed to AD 62.77%). Both groups utilized healthcare services with similar frequencies (remained MDD 9.65 days and progressed to AD 10.90 days).

Table 2 shows the performance of the MSM, MLP, and IPM on 20% held-out test data. In terms of predicting MDD patients who progressed to AD, the MSM outperformed the MLP across all the statistical measures except precision. However, the IPM showed improved model performance across all the statistical measures compared to the MSM or MLP. We observed that the IPM showed a higher precision in detecting patients who remained MDD than those who progressed to AD. Figure 4 shows the graphical representation of the area under the curve (AUC) measure of clinical relevance, depicting the differences between the predictive effectiveness of the MSM, MLP, and IPM. Only the IPM achieved substantial clinical relevance when considering the AUC measure above 80% (Verbakel et al., 2020; Carrington et al., 2022).

We evaluated the trained models to identify influencing features and their significant contributions to the IPM. Table 3 shows the ranking of features by importance to the ensemble model. We ranked the MSM features using the marginal structural model's weighted least square regression coefficients. Additionally, we ranked the MLP features by computing the feature importance based on the sum of absolute weights for each input feature connected to the first hidden layer of the neural network. Significant coefficients and weights were identified at p < 0.05. In determining significant variables, the results of the MSM-fitted model generate the coefficients and the corresponding p-values. However, for computing p-values for the MLP, we used an independent t-test to reject the null hypothesis that the mean of the feature weights is zero.

Four variables significantly influence the MSM model, with three of the four ranked as the top three features. They include, in order, any comorbidity diagnoses secondary to or co-occurring with MDD, residing in a rural location, age, and Medicare utilization days count after MDD diagnosis. On the other hand, race is the only significant feature in the MLP model, but it contains two of the four MSM top-ranked features in its top four ranked features (comorbidity and rural location). Thus, we show that the MSM features compensate for the bias in the IPM as they capture the underlying patterns in the data.

We performed an error analysis for the three models using the respective confusion matrices generated on the test data alone. Again, each model was trained on a separate training set independent of the test set. We used each model's confusion matrix to compute the models' overall precision, recall (sensitivity), and specificity. The IPM shows 89.8% precision (53 true positives and 6 false positives), showing a minimal rate of false positives compared to MSM's 71.2% (42 true positives and 17 false positives) and MLP's 52.5% (31 true positives and 28 false positives). However, the IPM has a slightly low recall, 75.7%, but 86.0% specificity compared to its precision value. MSM and MLP showed the same recall values at 66.7 and 66.0%, but the MSM has a moderately higher specificity (66%) than the MLP (57.6%). Thus, the IPM shows evidence of a low rate of false positives (wrong predictions) among patients who remained MDD while correctly identifying patients who are likely to progress to AD the majority of the time.

Tables 4A, B show the results of the Sensitivity Analysis (SA) performed on the imputed data of the original sample using the IPM and an MLP-only model. The IPM maintained its effectiveness and clinical relevance in predicting patients progressing to AD. The AUC clinical relevance measure remained above 80% (Figure 5A). Also, the IPM showed a high precision value of 82% despite the distortion in the data.

Furthermore, Figures 5A, B demonstrate how the MSM helps with bias removal in two ways: first, by modeling with the IPTW for estimating treatment effects. The MSM features comprise IPTW, which computes a stabilized weight that controls for time-varying confounding variables. We emphasize that using IPTW has been proven to eliminate bias compared to using ordinary variables as feature representation (Athey, 2015; Prosperi et al., 2020). Second, the sensitivity analysis confirms the effect of the estimated treatment effect on the IPM by removing the IPTW from the model and testing on the imputed dataset. Compared to the sensitivity analysis results with the IPM, the MLP-only model showed a lower AUC (Figure 5B). This shows the benefit of the removed IPTW features as a bias-removing mechanism.