Firstly, datasets were retrieved from the GEO database using the keyword “dementia with Lewy bodies” of Homo sapiens (organisms). Then, the inclusion criteria were used as follows: 1) datasets contained DLB patients’ and healthy controls’ miRNA expression profiles; 2) samples were blood samples; and 3) information on age and APOE 4 genotype were provided. Finally, only one dataset GSE120584 was screened out and downloaded. The normalized miRNA expression matrix, platform set, annotation file, and corresponding clinical information were downloaded and parsed via the GEOquery package (Davis and Meltzer, 2007). The average expression value was taken as the miRNA expression value for multiple probes corresponding to a miRNA. The GSE120584 dataset contained 1021 AD patients, 91 vascular dementia (VaD) patients, 32 mild cognitive impairment (MCI) patients, 169 DLB patients, and 288 healthy controls. In our study, we aimed to develop a signature for differentiating DLB patients from healthy controls. Therefore, only the miRNA expression profiles and clinical information of 169 DLB patients and 288 healthy controls were extracted from the GSE120584 to construct a DLB dataset for analysis. Detailed information of the DLB dataset is listed in Supplementary Table S1. Then, we used the train_test_split function from the scikit-learn’s model_selection package to stratified and randomly select 20% samples from the DLB dataset to form an independent test set (34 DLB patients and 58 healthy controls). The random state for train-test-split was 16. The remaining samples were taken to construct a training set (135 DLB patients and 230 healthy controls). The training and test sets are listed in Supplementary Tables S2, S3, respectively. The distribution of samples in datasets is listed in Table 1. No significant correlation was observed between the training and test sets in clinical characteristics.

In our study, three steps were performed to identify the qualitative signature for DLB, which were described as follows: 1) Feature generation. Given that the expression values of a miRNA pair (i, j) are denoted as Ei and Ej. The REO pattern of the miRNA pair is denoted as 1 (or 0 or −1 ) if Ei > Ej (or Ei = Ej or Ei < Ej). We calculated the values of the REO patterns for all miRNA pairs in each sample. The REO patterns of all miRNA pairs were used as new features for feature selection. 2,547 miRNAs constructed 3242331 miRNA pairs. 2) Feature selection. All feature selection methods were run on the training set. One miRNA pair was defined as a reversed stable miRNA pair when its REO pattern was identical in most control samples and was opposite in most patient samples. We first quickly identified 962 reversed stable miRNA pairs by setting the threshold at 60%. Then, the random forest (RF) was used to select 400 top-ranked important reversed stable miRNA pairs. The RF was implemented by the RandomForestClassifier function of the scikit-learn’s ensemble package. The random state was 16, and all other parameters were kept at default. Lastly, the support vector machine-recursive feature elimination (SVM-RFE) (Sanz et al., 2018) with stratified-3-fold cross-validation (SVM-RFE-CV) was applied to extract an optimal miRNA pair subset from 400 top-ranked important reversed stable miRNA pairs. SVM-RFE-CV was implemented by the RFECV function of yellowbrick’s model_selection package (Bengfort and Bilbro, 2019). Linear SVM was used as the base classifier. The penalty parameter of the error term is set to 1. All other parameters were kept at default. 3) The signature construction. According to the reference (Shigemizu et al., 2019), two clinical factors, age, and APOE ε4 genotype, may help to differentiate DLB patients from healthy controls. Therefore, we constructed the qualitative signature by combining the optimal miRNA pair subset and two suggested clinical factors. The numerical values of age were mapped to three classes (−1, 0, and 1) according to the thresholds at 70 and 80.

Two commonly used machine learning methods, logistic regression (LR) (Sperandei, 2014) and SVM (Noble, 2006), were employed to build prediction models for discriminating DLB patients from healthy controls. Detailed principles of LR and SVM have been provided in previous papers (Noble, 2006; Sperandei, 2014). Thus, we only introduced the construction of our models in detail here. After identifying the optimal miRNA pair subset (17 miRNA pairs), we constructed new training and test sets by calculating the REO pattern’s values of these miRNA pairs. For comparison, we established four prediction models (DLB1, DLB2, DLB3, and DLB4) by LR and SVM based on 17 miRNAs pairs and the signature (17 miRNA pairs and two clinical factors), respectively. DLB1 and DLB2 were constructed with 17 miRNAs, while DLB3 and DLB4 were built with the signature. The training and test sets for four models are listed in Supplementary Tables S4, S5, S6, S7. SVM was implemented by the SVC function of scikit-learn’s SVM package. For all SVM models, the parameter probability was set to true. Given that the parameter gamma, penalty parameter of the error term, and kernel function are crucial for SVM models, we conducted a grid search to find their optimal values. All other parameters were kept at default. Moreover, LR was implemented by the LogisticRegression function of scikit-learn’s linear_model package. For all LR models, the parameters max_iter and penalty were set to 10000 and l2, respectively. The inverse regularization parameter was also tuned by grid search. All other parameters were kept at default. The grid search was implemented by the GridSearchCV function of scikit-learn’s model_selection package. Detailed information concerning search space and optimum values is summarized in Table 2. Here, all prediction models were validated using internal stratified-3-fold cross-validation and external test set techniques.

Sensitivity (SE), specificity (SP), overall prediction accuracy, F1 score, and area under the receiver operating characteristic (ROC) curve (AUC) were calculated. ROC curves of the models were also plotted.

We used the limma package to identify significantly dysregulated miRNAs of 21 miRNAs. Corrected p-value < 0.05 was considered significant. The miRWalk 3.0 online database (http://mirwalk.umm.uni-heidelberg.de/) and the mirDIP online database (http://ophid.utoronto.ca/mirDIP/) were used to predict target genes of these dysregulated miRNAs. TargetScan, miRDB, and miRTarBase datasets were incorporated into the miRwalk framework (Sticht et al., 2018). The cross-part between the genes identified by miRWalk and mirDIP were then extracted as target genes. Based on these target genes, the STRING 11.0 (https://string-db.org/cgi/input.pl), an online tool for retrieving interacting genes, was applied to construct a protein-protein interaction (PPI) network. The confidence score threshold was set to 0.9. Then, the CytoHubba (Chin et al., 2014), a well-known plugin of Cytoscape, was employed to identify hub genes. The eccentricity algorithm was selected, and all other plugin parameters were left at their default values. The five top-ranked genes were chosen as hub genes. Lastly, the ClusterProfiler (Yu et al., 2012), a widely used R package of Bioconductor, was used to perform gene ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analyses.

Figure 2 shows the automatic tuning of the number of features (miRNA pairs) selected by SVM-RFE-CV. From Figure 2, we observed that the highest accuracy, with 71%, came from 17 features. After 17 features, as non-information features are added to the model, the accuracy decreased gradually. After about 150 features, it entered a relatively steady state. Therefore, these 17 features were selected as the optimal feature subset, as shown in Table 3. These 17 miRNA pairs contain 21 miRNAs. Among them, there are nine significant dysregulated miRNAs (corrected p-value < 0.05). The expressions of hsa-miR-92a-2-5p and hsa-miR-6813-5p were downregulated in DLB patients with healthy controls. And the expressions of 7 miRNAs, including hsa-miR-5698, hsa-miR-5001-3p, hsa-miR-5195-3p, hsa-miR-4687-3p, hsa-miR-4793-5p, hsa-miR-1322, and hsa-miR-4756-3p, were upregulated. The remaining 12 miRNAs, including hsa-miR-3148, hsa-miR-335-3p, and hsa-miR-3677-3p, etc., were not significantly altered in DLB patients and healthy controls. More details are given in Supplementary Table S8.

Based on 17 miRNA pairs and the signature, we built four models by SVM and LR, namely DLB1, DLB2, DLB3, and DLB4, respectively. These established models were firstly evaluated by the stratified-3-fold cross-validation method. The cross-validation results of the four models are shown in Table 4; Figure 3. From Table 4; Figure 3, we found that the DLB3 had the highest average sensitivity, F1 score, accuracy, and ROC AUC in the four models. The average sensitivity, F1 score, accuracy, and ROC AUC of the DLB3 are 72.59, 84.13, 84.39, and 90.26%, respectively. In the stratified-3-fold cross-validation of the DLB3, the ROC AUCs of two validation sets were as high as 93.39 and 91.56%, and the other one achieved 86.09%. However, the DLB1 provided the highest average specificity of 92.63%. It was higher than 90.89% of the DLB2, 91.31% of the DLB3, and 90.45% of the DLB4. The difference in the average specificity of the four models was slight, no more than 3%. Although the DLB1 provided better prediction in terms of specificity, it had lower sensitivity, F1 score, accuracy, and ROC AUC than the DLB3. In general, the DLB3 showed the best performance in the training set, and the DLB4 was second. Meanwhile, we observed that, except specificity, DLB3 and DLB4 performed better than DLB1 and DLB2 in terms of the other four evaluation criteria.

Then, an independent test set was used to evaluate the predictability of these four models. Prediction results for the test set are shown in Table 5; Figure 4. Similar to the prediction results of the training set, DLB3 and DLB4 performed better than DLB1 and DLB2 in terms of sensitivity, F1 score, accuracy, and ROC AUC. Especially for sensitivity, compared to the DLB1 and DLB2, the sensitivities of the DLB3 and DLB4 improved by more than 20 and 17%, respectively. Sensitivity is of great importance within a diagnostic rule-out approach. The sensitivity, specificity, F1 score, accuracy, and ROC AUC of the DLB3 were 64.71, 87.93, 78.95, 79.35, and 87.32% in the test set. For the DLB4, they were 67.65, 91.38, 82.19, 82.61, and 87.63%, respectively. The DLB3 had the lowest specificity of 87.93%, while the other three models' specificity was 91.38%. In addition, unlike the training set results, the DLB4 was relatively more superior to the DLB3 in the test set under the evaluation of each evaluation criterion.

Overall, among these four models, the DLB3, which was developed by SVM based on the signature, outperforms other models in the training set. However, the DLB4 constructed by LR based on the signature provides the best prediction in the independent test set. Comparatively speaking, it is more important to classify external samples outside of the training set correctly. Therefore, in our study, the DLB4 is suggested to discriminate DLB patients from healthy controls. Meanwhile, we noted that the signature-based models perform better than 17-miRNA-pairs-based models in the training and test sets. These results indicate that integrating the clinical factors (age and APOE ε4 genotype) and 17 miRNA pairs improves the prediction performance.

Firstly, 328 genes were predicted as target genes of these nine miRNAs by miRWalk and mirDIP (Figure 5A). Detailed information on these target genes is provided in Supplementary Table S9. A PPI network was established with 107 nodes and 173 edges by the STRING database (Supplementary Figure S1). The hub genes selected from the PPI network are shown in Figure 5B. The five highest-scored genes, including SRF, MAPK1, YWHAE, RPS6KA3, and KDM7A, were chosen according to the eccentricity scores. GO analysis revealed that 328 target genes were enriched in 11 terms, including cell junction assembly, synapse organization, protein methylation, protein alkylation, etc., as shown in Figure 5C. KEGG pathway analysis indicated that they were enriched in 2 pathways, including the apelin signalling pathway and insulin resistance (Figure 5D). More detailed GO and KEGG enrichment analyses are listed in Supplementary Tables S10, S11.