For proper evaluation of the proposed model, a number of standard performance metrics have been used. The most intuitive performance measure is accuracy; however, for a highly imbalanced dataset (which is the case here), accuracy is not reflective of true performance. Therefore, various evaluation parameters, such as sensitivity, specificity, and Matthew’s correlation coefficient (MCC) are reported. Youden’s index and balanced accuracy are also considered to be important evaluation metrics for imbalanced data and are, therefore, extensively explored in this research.

To design the proposed method, we used the benchmark dataset provided in Kandaswamy et al. (2013). The dataset consists of 445 ECM proteins and 3,327 non-ECM proteins. The 445 ECM proteins were curated from Swiss-Prot release 67 by first filtering 1103 ECM proteins from the pool of 17,233 metazoan-secreted protein sequences. Similarly, the negative dataset of 16,130 proteins were curated from secretory proteins that are annotated as non-ECM. Later, 445 ECM and 4,187 non-ECM nonhomologous sequences were further filtered out with the help of a clustering method (Li et al., 2001) by removing the sequences which showed 70% or higher similarity.

Feature representation ability of the CKSAAP improves with large values of the parameter k, which is expected to result in a more robust model (Park et al., 2020b; Usman and Lee, 2019; Wu et al., 2019; Chen et al., 2017). However, the model utilizing a large number of features is susceptible to noise, resulting in a degraded performance. Furthermore, training the model on a large number of features not only results in an increased training time and complexity but is also prone to overfitting. To which end, feature selection/engineering, which involves the selection of most significant features, has to be employed. Feature selection techniques are broadly categorized into two types: 1) supervised methods, which remove the irrelevant features based on a target variable, and 2) unsupervised methods, which use correlation techniques to remove redundant information. A number of methods for feature selection have been proposed in the literature, including minimum redundancy maximum relevance (mRMR) (Peng et al., 2005), student’s t test (Student, 1908), info-gain (Mitchell et al., 1997), and generalized variant of strictly standardized mean difference (GSSMD) (Park et al., 2020a). Another useful method is to map the original data into a lower-order dimensional space through some transformation function. The eigen-space transformation or the principal component analysis method (PCA) (Jolliffe, 1986) is considered to be the benchmark method in this context. Other approaches such as an independent component analysis (ICA) (Comon, 1994), a kernel principal component analysis (KPCA) (Schölkopf et al., 1998), uniform manifold approximation and projection (UMAP) (McInnes et al., 2018), and t-distributed stochastic neighbor embedding (t-SNE) (Van der Maaten and Hinton, 2008) are also being successfully used to deal with the curse of dimensionality.

Most of the techniques mentioned above are unsupervised in nature. To address this issue, we propose to use a novel approach called a deep latent space encoding (DeepLSE) classifier for the latent space encoding based on an auto-encoder. Latent space refers to the representation of compressed data in which similar points would be in a close group, as shown in Figure 5. Similar samples tend to have common significance, which can be packaged into the latent space representation of the raw data. Thus, as the dimensions are reduced, the redundant information from the input samples is removed, leaving only the most important features of the data. In other words, the method can learn a compact representation of feature space and remove the noisy or potentially confusing information which is good for both the classification and reconstruction tasks. This ensures that the encoded features truly represent the sample information. The DeepLSE method has been found to be an impressive method for the feature space reduction and has outperformed other approaches in relatively similar tasks such as AFP-LSE (Usman et al., 2020) and E3-targetPred (Park et al., 2020b). The architecture of the proposed method is depicted in Figure 2 named as ECM-LSE.

The workflow of the proposed study is aimed to obtain the best classification model based on two variables, that is, the gap between the two amino acid pairs and the number of units in the latent space LVs. An ablation study has been designed to acquire models with varying number of aforementioned variables and is depicted in Figure 3 (a). The samples are distributed into training, validation, and test datasets as discussed in the Subsection 2.2 and are encoded with incrementing values of k from 0 to 10. The resultant features are used to train the model with incrementing values of the latent space variables ranging from 2 to 9. As discussed earlier, for each configuration, 20 independent trials are performed and the mean results are computed. A consistent procedure is repeated for all 1,760 trials of the 88 unique model configurations. The model with the best average results is finally selected as the base model to perform prediction and is named as ECM-LSE. In Table. 1, the average results of the balanced accuracy have been reported. It can be observed that the model with values of gap k = 8 and latent variables LV = 7, accounts for the best. The results for the rest of the evaluation parameters are illustrated in the form of surface graphs in Figure 4.

The performance of the proposed model is compared to the benchmark approaches and the findings are reported in Table 2. For a fair comparison, only the best reported results of the respective approaches are presented. The performance of the proposed ECM-LSE is compared with the contemporary methods including EcmPred (Kandaswamy et al., 2013), a sparse learning approach for the prediction of ECM (ECMSRC) (Naseem et al., 2017), and PECM (Zhang et al., 2014). In particular, the reported sensitivity, specificity, MCC, Youden’s index, and accuracy on the benchmark dataset of EcmPred (Kandaswamy et al., 2013) are compared.

The results clearly show that the proposed method has better balanced accuracy as compared to the contemporary approaches. In particular, the proposed ECM-LSE method achieves the highest sensitivity of 84.14% outperforming the best competitor (PECM) by a margin of 10.91%. The specificity value achieved by the proposed ECM-LSE also compares favorably with other methods, which confirms the balanced unbiased learning effect. It is noteworthy to point out that the accuracy metric cannot provide true fitness of the models given the skewed distribution of test dataset toward the negative (non-ECM) class. Any model with all negative predictions can achieve 100 × 2247 145 + 2247 = 93.94 % accuracy easily. As discussed in Subsection 2.1, the parameters of balanced accuracy, MCC, and Youden’s index are considered more reliable in the case of imbalanced dataset. Therefore, despite achieving 86.35% test accuracy, which is 0.17% lower than the PECM, better balanced accuracy and Youden’s index values, which is 3.93% and 0.08 units higher, respectively, demonstrate the superiority of the proposed method. Similarly, the MCC value achieved by ECM-LSE is 7.63% higher than the PECM method. MCC metric is preferred for accuracy and is considered as more reliable statistical parameter because it produces a higher value only if the classifier achieved good results in all four categories of the confusion matrix (Chicco and Jurman, 2020). In general, the proposed ECM-LSE approach has shown to comprehensively outperform the contemporary methods in all aspects of balanced and unbiased prediction performance.

Furthermore, unlike contemporary methods where handcrafted embedding schemes are utilized for separately developing the classifier and the feature extraction modules, the proposed ECM-LSE method learns directly from the original feature space. The LSE encoding effectively learns the distinguishing characteristics of classes in a lower dimensional feature space and allows the visualization of proteins sequences. This aspect of ECM-LSE is further explained in Section 3.4.

To verify the practical usefulness of our method, herein, we perform the validation of our method on experimentally verified ECM proteins. In particular, we collected 20 experimentally verified human ECM proteins from UniProt (Consortium, 2018). The collected sequences were not present in the positive or negative datasets of ECM-LSE. The criteria for the selection were based on the clear experimental evidence in the literature for the given sequence entry. We evaluated the EcmPred (Kandaswamy et al., 2013), ECMSRC (Naseem et al., 2017), PECM (Zhang et al., 2014), and ECM-LSE methods. As shown in Table 3, ECM-LSE (k = 8 and LV = 7) correctly identified 19 proteins as extracellular matrix proteins, whereas PECM, ECMSRC, and EcmPred identified 18, 16, and 15 proteins, respectively. It is noteworthy to point out that the models were trained on ECM proteins from metazoans; therefore, the superior performance of the proposed ECM-LSE on proteins from a completely different organism suggests that it can be effectively utilized for the annotation of unknown proteins.

For typical classification problems such as lysine acetylation site prediction in proteins (Wu et al., 2019) or the identification of protein–protein binding sites (Fernandez-Recio et al., 2005), a large number of positive and negative samples are usually available in the datasets. Therefore, the problem of class imbalance or intra-class variation is not a major concern (Johnson and Khoshgoftaar, 2019). However, the limited availability of ECM samples results in an imbalanced dataset, resulting in an ill-posed problem. A number of approaches, including sample rescaling, have been proposed in the literature to tackle the imbalanced data problem (Xiao et al., 2016; Kabir et al., 2018). Classifiers based on these rescaling techniques tend to behave well; however, the generalization of the method is compromised. Furthermore, the comparison of methods using rescaled samples with the methods using a standard dataset is not reasonable. In the proposed study, we utilize a standard dataset and develop a method that effectively discriminates the ECM proteins from non-ECM. This is achieved through the latent space learning of the CKSAAP features. For better understanding, we compare the t-SNE projection of the CKSAAP features with the proposed latent space in Figure 5.

For visualization purposes, the data were projected on two dimensions using t-SNE (Van der Maaten and Hinton, 2008) projection of the original feature space and two variable latent spaces in the case of ECM-LSE. In the t-SNE projection shown in Figure 5A, it can be observed that both ECMs and non-ECMs appear in an overlapping fashion, suggesting that the development of the ECM classifier using original feature space is an arduous task. As shown in Figure 5B, the proposed latent space encoding (ECM-LSE) presents superior learning capabilities and maps the ECMs and non-ECMs in separate regions in contrast to the unsupervised subspace learning method of t-SNE (Van der Maaten and Hinton, 2008).

The proposed method, as shown in Figure 5B, tends to form distinguishable clusters of ECM and non-ECM proteins. Although some overlap can be observed in the projection of the proposed method, it is still remarkably better than that of the t-SNE, and since the projection is shown for two latent variables only, the actual model with seven latent variables is expected to mitigate the overlap to a greater extent. These projections are also helpful in understanding the working principle of the proposed method and the motivation for the development of nonlinear auto-encoded learning of latent space.

The proposed hybrid approach presents a hybrid design with capabilities of efficient feature selection and classification of ECM proteins. The latent space dynamically reduces the dimension of the feature space and retains only the relevant information sufficient to efficiently distinguish ECM from non-ECM samples. Although, the proposed method can predict ECM from different organisms, it is not a replacement for gold standard wet lab–based testing. Furthermore, due to the scarcity of available ECM proteins the model may show biased performance in favor of already explored ECM and finding novel proteins may require the fusion of additional information. However, efforts have been made to avoid overfitting in order to seek the generalization property of the model by deploying dropout and batch normalization techniques. Further enhancements to the ECM prediction task where scarcity of the positive samples persists can be made by applying a transfer learning approach, where a large scale model is trained on a closely related dataset and is further fine-tuned for ECM samples. The Python implementation of the proposed algorithm has been made public, and interested users can utilize the algorithm for their problem of interest. The algorithm is available at (https://github.com/Shujaat123/ECM-LSE/blob/master/ECM_LSE_Online.ipynb). In the future, we aim to explore the efficacy of the auto-encoder–based classifiers on other bioinformatics problems.