Termite fungi are valuable but endangered, and urgent research and conservation efforts are needed. However, data on ITS sequences for termite fungi in GenBank are incomplete, making it crucial to synthesize data from different sources. In this article, ITS sequence data from two GenBank, BOLD and NCBI, was compiled by us. Specifically, 101 ITS sequences were obtained from BOLD, with the number of sequences for each genus ranging from 1 to 12. At NCBI, 1740 ITS sequences were obtained, with the number of sequences for each species ranging from 1 to 799. After synthesizing the ITS sequence data from these two GenBank and removing termite fungal species with fewer than 7 sequences, 1704 sequences belonging to 17 termite fungal species were obtained. The labels of each termite fungal species are presented in detail in Table 2. This data can be used for further research and conservation efforts for these valuable and endangered fungi.

The ITS region of termite mushrooms collected from Binh Duong province, Vietnam, was sequenced, and the resulting sequences have a length ranging from 669 to 1050 base pairs. These termite mushroom samples have a morphology similar to that of Termitomyces clypeatus, Termitomyces microcarpus and Termitomyces striatus. The sequence data for these eight termite mushroom samples have been published and stored in the NCBI GeneBank. For more detailed information about these termite mushroom samples, please refer to Table 3.

The extraction of features from biological sequences is a crucial step in computational biology. Biological sequences are typically composed of a string of letters, which must be converted into numerical vectors before they can be utilized in machine-learning algorithms (Kamath et al., 2014). The K-mer feature technique has been employed to represent information for ITS sequences to classify species based on barcodes, as demonstrated by previous studies (Schloss et al., 2009; Deshpande et al., 2016). In 2016, Delgado-Serrano utilized K-mer encodings to transform ITS sequences into numerical vectors. The accuracy of the prediction model was affected by the size of the K-mer utilized (Delgado-Serrano et al., 2016). In our proposed approach, a combination of K-mer and CountVectorizer techniques was employed to encode ITS sequences into numerical vectors. An illustration of the methodology utilized to digitize sequence information is presented in Figure 2.

In Figure 2, The process of digitizing ITS sequences has been illustrated. This process is similar to that of using Natural Language Processing (NLP) tools from Sklearn to convert our K-mer words into numerical vectors. These vectors, which represent the count of each K-mer in the vocabulary, have the same length as unigrams.

Supervised machine-learning techniques are widely used in computational biology to solve various problems. Several traditional machine-learning algorithms such as k-nearest neighbors, Naïve Bayes, and decision trees have been successful in identifying mushroom species based on barcode data (Schloss et al., 2009; Delgado-Serrano et al., 2016; Deshpande et al., 2016). However, these models have relatively low accuracy. In our research, two solutions were tested: i) The first set utilized well-known classification methods like Naïve Bayes and Random forest to predict the names of termite mushroom species; ii) In the second, automated models for predicting termite mushroom species with higher accuracy were built by us using Ensemble learning algorithms such as XGBoost and CatBoost.

Gradient Boosting Decision Trees (GBDT) (Friedman, 2001) is a method that uses decision tree ensembles to predict target values. A GBDT is constructed by splitting observations based on the attribute values of the input data. The model can find the best way to divide data and determine the most time-consuming part of the partitioning process. To build a GBDT model with T trees from a dataset consisting of n samples, the prediction process according to the GBDT method is as follows: y ^ i 0 = 0 y ^ i 1 = f 1 x i = y ^ i 0 + f 1 x i y ^ i 2 = f 1 x i + f 2 x i = y ^ i 1 + f 2 x i . . y ^ i K = ∑ k = 1 K f k x i = y ^ i K − 1 + f K x i (1) where y ^ i K is the predicted value of the i ^th sample at the k ^th iteration

The cost function of GBDT has two parts: a training error and regularization, as follows: Cost = ∑ i = 1 n l y i , y ̂ i + ∑ k = 1 K Ω f k (2) where Ω f k = γ T + 1 2 λ w 2 , ∀ k = 1 , K ¯ . T is the number of leaf nodes, w is the score for a leaf node, γ is the leaf penalty coefficient, and ensures that leaf nodes’ scores are not too large.

CatBoost is a viable option for gene sequence data analysis, as indicated by recent research (Robson, 2022). In our experiments with termite mushroom data, it was observed that CatBoost performed comparably to XGBoost in terms of prediction accuracy. However, a relatively longer training time is required by CatBoost than that of XGBoost to achieve a similar level of performance. Therefore, XGBoost was chosen as the primary algorithm for our prediction model.

The XGBoost model’s performance depends on several key parameters such as 'max_depth', 'gamma', 'n_estimators', and 'learning_rate'. These parameters are known as hyperparameters and can be adjusted manually during training or automatically. The proposed enhanced model uses the Bayesian Optimization technique (Klein et al., 2017) specifically Random search, to tune the hyperparameters. Bayesian Optimization was applied to tune the four main parameters of the XGBoost classifier: 'max_depth', 'gamma', 'n_estimators', and 'learning_rate'.

To improve the predictive performance of the model, cross-validation with k = 5 was performed to select the best classification model, in addition to using Bayesian Optimization to tune hyperparameters. A new dataset, which consisted of n data samples and m features, was obtained from the results of phase 1. An optimization parameter was then used as input for Algorithm 1 to build an optimal classification model.

Algorithm 1. Building the best XGBoost classifier

Input: D T e r = x i , y i ∈ R m × 0,1 , ∀ i = 1 , n ¯ ; hyperparameter is Ɵ = {'max_depth': int(max_depth), 'gamma': Gama, 'n_estimators': int(n_estimators), 'learning_rate':learning_rate }

Our study has developed an automated process consisting of four stages to predict the species name of a new termite fungus. The first stage involves collecting termite fungus data from ITS gene sequence repositories. In the second stage, sequence features are extracted and encoded. The third stage involves building a classifier by constructing and tuning parameters to find the optimal classifier. Finally, in the fourth stage, the classifier is used to predict new termite fungus samples. Figure 3 provides a detailed description of this process.

In our study, the terms “true” and “false” predictions can arise from the model’s misclassification or failure to predict accurately, such as false negatives or false positives, or other concepts applied to the prediction targets. Specifically, the phrase “predicting the species of Termitomyces” is referred to as a true positive (TP), while the phrase “correctly excluding the species of Termitomyces” is referred to as a true negative (TN). On the other hand, the phrase “predicting the species of Termitomyces incorrectly” is designated as a false positive (FP), and a “missed or misclassified prediction” is considered a false negative (FN). These conditions are utilized as stopping points during initial data training. To evaluate the performance of our proposed model, various methods were applied to assess its machine-learning abilities on DNA sequence data (Gupta, P., et al., 2021). These methods include the following:

❖ Accuracy: The proportion of correctly predicted cases is known as accuracy, and it can be calculated using the following formula:

In the experimental process, Python 3.9 and the libraries Scikit-learn, Biopython, XGBoost, CatBoost, and Bayesian optimization were employed to construct a mushroom classification model following the proposed process depicted in Figure 3. The results of each stage a, b, c, and d are attached.

❖ During stage a: Data was collected through the following steps: (a.1) retrieving data from the NCBI and BOLD GenBanks, which yielded 1740 sequences of 28 mushroom species; (a.2) selecting 17 species that had at least 10 sequences per species.

The accuracy of predictive models based on sequence data is significantly impacted by the size of K-mers (Delgado-Serrano et al., 2016). To explore this impact, a study was conducted using different K-mer lengths, which resulted in varying classification accuracies. The sequence in Figure 3 was used to build a classifier, with machine-learning algorithms such as Naive Bayes (MultinomialNB), RandomForest, XGBboost, and Catboost. The classifier’s results for each K-mer size are presented in Figure 4. We found that each algorithm produced different predictive results (accuracy) for each K-mer size. Specifically, Catboost produced results ranging from 0.87–0.88, XGBboost had results from 0.88–0.91, and RandomForest yielded results from 0.87–0.89. However, the Naive Bayes (MultinomialNB) model had the lowest accuracy, ranging from 0.59–0.61. The classifier’s results for each K-mer size are presented in Figure 5.

Figure 6 presents detailed information on the impact of K-mer size on the highest accuracy achieved by each algorithm. Notably, the XGBoost algorithm achieved the highest classification accuracy when the K-mer size was set to 7, which was also used to build the automated model for predicting mushroom species names.

Apart from using accuracy as a measure of the classification model’s performance, other metrics such as precision, recall, F1 score, or AUCROC are also utilized to evaluate the classifiers’ performance. A summary of the performance of the surveyed machine-learning algorithms is presented in Table 4.

Furthermore, the AUCROC for each class was calculated using the ROC curve method with the OvR macro-average for the multi-class model utilized (Pedregosa et al., 2011). In this study, the last class (class 16) was designated as the positive class, while all other classes were considered negative classes. The visual representations of each class’s results are presented in Figure 6.

Previous models for predicting fungal species accuracy have been evaluated using the K-mer method and machine-learning techniques such as k-Nearest Neighbor, Naïve Bayes, and Random Forest, with results presented in Table 5. Our proposed approach demonstrates superior performance when utilizing a K-mer size of 7 with the XGBoost classification algorithm. Table 5 presents a comparison of various classifiers' performance for predicting fungal species using ITS sequences.

The ITS sequences of termite fungi collected from Binh Duong province, Vietnam, were published on NCBI and are detailed in Table 3. Our proposed classification model predicted species identification with comparable results to those obtained from NCBI. For instance, sequences MF163150-BD1, MF163151-BD2, and MF163147-BD7 were identified as the same species as those on NCBI. Moreover, the species identification of MF163149-BD4 was consistent with the identification on NCBI. However, for MF163445-BD3, MF163446-BD6, and MF163149-BD4, the identification was previously unknown or unclear. Our proposed classification model successfully identified MF163445-BD3 and MF163446-BD6 as Termitomyces striatus, consistent with the type strain of the collected fungi. The results for MF163149-BD4 were also consistent with the species identification on NCBI. Table 6 presents the details of the species identification results.

Accurately identifying new species is crucial for studying biodiversity and formulating conservation policies for endangered species (Van Velzen et al., 2012). Traditional methods of species identification based on physical characteristics can be difficult, prompting the use of DNA barcoding as an alternative approach (Hibbett et al., 2011). In this study, a novel computational method is proposed that utilizes K-mer techniques and NLP vectorization to convert DNA barcode sequence data into digital features. The XGBoost algorithm is then employed to build a model capable of predicting termite mushroom species using the ITS sequence as a DNA barcode.

The performance of the developed model was evaluated on 1704 sequences of 17 mushroom species obtained from two ITS GenBanks. The evaluation was conducted using standard classification metrics such as accuracy, precision, recall, F1-score, and AUCROC.

Our proposed model was assessed by comparing its predictions with the species identification results on NCBI, demonstrating complete consistency with the identified species of the ITS sequences of mushrooms, as well as predicting the species names of two sequences that had not previously been identified. An example of this is the Termitomyces striatus mushroom specimen found in Binh Duong province, Vietnam, which was correctly identified by our proposed model. Furthermore, when compared to four other research groups' machine-learning models for predicting termite mushroom species names, our proposed model achieved an accuracy of 0.91 and an average AUCROC score of 0.99, demonstrating its efficacy in species identification. These results suggest that our proposed model is a valuable tool for identifying termite fungi species in Binh Duong province, Vietnam, and could be applied to other mushroom species as well.