Feature selection is a technique that is used to reduce the dimensionality of the data set, which leads to better learning performance and/or lower computational cost. This technique selects the most relevant attributes in the data set by removing noisy, irrelevant and redundant features (11). Different feature selection techniques can be found in the literature, and can be categorised according to the search strategy. There are three main approaches: filter, wrapper, and embedded (11).

In this work, the wrapper approach is used, since it makes use of a learning algorithm to determine the best subset of attributes, called features, where an evaluation is usually made in terms of predictive accuracy. Due to the use and dependence of a learning model, this type of approach can become computationally expensive, though the possibility of selecting irrelevant features is less likely (12). Among the wrapper techniques, we used the SFA. This technique has four different types, and each type differs in the way it selects or removes features from the data set: sequential forward selection (SFS), sequential backward selection (SBS), sequential forward floating selection (SFFS) and sequential backward floating selection (SBFS).

Grid search is an exhaustive search technique for setting hyperparameters of a given model. With it, it is possible to analyse the results of a ML model, and then decide which configuration best fits the target problem. According to Bergstra and Bengio (13), despite having limitations, this technique is widely used along with the manual search technique.

ML is a branch of artificial intelligence that is composed of several techniques that have been widely used for pattern learning (8, 14–18). The ML models used in this work are Random Forest (RF), Adaptative Boosting (Adaboost), Gradient Boosting Machines (GBM), eXtreme Gradient Boosting (Xgboost), k-Nearest Neighbours (KNN), Naive Bayes (NB) and Multilayer Perceptron (MLP).

The following metrics are used: accuracy, precision, sensitivity and F1-Score. With the exception of accuracy, in the other metrics, the value of the metric in each class and the macro average of each one of them is also analysed.

In this work, data regarding Dengue and Chikungunya notifications from the state of Amazonas and the city of Recife, Pernambuco from 2015 to 2020 are used. Regarding the state of Amazonas, data were retrieved from the Health Problem and Notification Information System, from Portuguese Sistema de Informação de Agravo de Notificação (SINAN) ⁵ . SINAN is the official system for disease reporting in Brazil. Diseases from the national list of compulsory notification must be reported, and this list includes Dengue and Chikungunya. This data set contains 57,445 entries and 146 variables and hereafter is referred to as “SINAN-db”.

The data set for Recife was retrieved from an open data set named Portal de Dados Abertos do Recife (19), maintained by the Recife Health Department, whose primary source is also the SINAN, and therefore it follows the same dictionary pattern, and allows integration without further issues. This data set contains 83,073 registers and 124 variables and is referred to as “Recife-db” in this work.

Figure 1 illustrates the steps taken during the pre-processing of the data set. First, both data sets were integrated. Variables available in only one of the data sets were disregarded. The resulting data set from the integration of SINAN-db and Recife-db has 140,518 registers and 120 variables.

The output classes were grouped into three distinct classes:

Only records confirmed or denied by clinical diagnoses were selected. Registers that did not relate signs or symptoms were discarded since they are the most important information for classification models. Moreover, variables with more than 50% of data missing were also removed. Besides the original variables, a new one (DIAS) was created so that the time (in days) from onset of these symptoms to the date of notification could be added to the models. For the selection of attributes, specialists were consulted. After coding variables as numbers, duplicates were removed, and missing values were replaced by “not informed” for each variable. Registers with missing values for all variables were also removed. Finally, the clean data set consisted of 17,948 registers in the DENGUE class, 5,724 in the CHIKUNGUNYA class and 16,704 in the OTHERS class, totalling 40,376 registers with 27 variables. In data science, a higher number of registers of a specific class compared to another in the same data set is known as imbalance and it can bias the ML model, which favours the classification of the class that has the largest number of registers (20).

In order to balance the data set, the random undersampling technique was performed. In this technique, the class with the least number of registers defines the amount of the other classes, so that all classes have the same number of registers. After balancing, the data set still had 27 attributes and 17,172 records, with 5,724 for each of the three classes. The 27 variables resulting from the pre-processing are described in Table 1 . The data set can be accessed in Mendeley Data (21).

The experiment is divided into three main steps: (a) optimisation of hyperparameters and attribute selection, using Grid Search and SFA; (b) evaluation of models performance; and (c) specialist evaluation.

Table 4 presents the results from the Grid Search technique of the seven models: Adaboost, RF, GBM, Xgboost, KNN, MLP and NB.

Regarding SFA, the techniques that presented the best performance were SFFS and SBS. The size of the subset of attributes ranged between 10 and 20 attributes, and the most common attributes were CS_RACA, CS_ZONA, FEBRE, EXANTEMA, NAUSEA, ARTRALGIA, DOR_RETRO, which appeared in all subsets. Table 5 shows the attributes selected by the SFA techniques for each model.

The model that best performed was the Xgboost model, using the SFFS technique with 20 attributes (the largest subset size in this experiment), eta = 0.3 and max_depth = 2, which obtained 62.3% accuracy. On the other hand, the KNN model with 19 attributes, selected by the SBS technique, metric = euclidian, n_neighbors = 2 and weights = uniform, was the worst model in the grid search, with 57.39% accuracy.

Table 6 presents the results of accuracy and macro medians from recall, precision and F1-score. The GBM model outperformed all the models. It is interesting to note that the MLP model showed poor performance in comparison with the result it presented in the grid search. This difference may indicate that the MLP model failed to generalize the data during training and underfitting probably occurred and, as consequence, the MLP model did not performed well when using the test set.

The results of the CHIKUNGUNYA class are presented in Table 7 . This class obtained the highest results, achieving more than 80% recall in KNN and MLP, although, those same models demonstrated the lowest values of precision. The other models had a better balance between these two metrics. For the F1-score metric, the situation is very similar, with the only difference being that the MLP outperformed the NB. In general, the GBM model obtained the best results.

The results for DENGUE class are presented in Table 8 . Recall values were below 50% for all models, which were the lowest values. The results of the precision and F1-score were not much better either, i.e., below 60%. Overall, the GBM model obtained the best results again.

Table 9 presents the results of recall, precision and F1-score for the OTHERS class. Results were similar and were around 60%, with the exception of the KNN model, which had a considerable drop in recall (34.83%), and the MLP and NB models, which also showed poor performance regarding the precision metric (51.49% and 54.32%, respectively). In this class, none of the models stood out, and GBM, Xgboost and NB models obtained the best values for recall, precision, and F1-score, respectively. GBM and Xgboost model were considered the best for classifying this class.

The grid search results did not present a large variation, ranging from 57% to 62% accuracy. In addition, none of the models presented accuracy above 70%, which shows the difficulty involved in classifying arboviral diseases using only clinical and socio-demographic data.

It was possible to observe that the DENGUE class was the class with the lowest performance, thus highlighting the difficulty in classifying this disease with the data used. However, there are some reasons that may explain this: (a) the classification is a multi-class task, which makes it difficult to generalize the three classes;(b) the applying of the undersampling technique to balance the data may also have affected the DENGUE class, as at the end of the pre-processing there were almost 18,000 Dengue registers that could have been used, but this number had to be reduced to almost 6,000 due to the low amount of Chikungunya registers; (c) lack of laboratory attributes often associated to Dengue such as leucocyte count, haematocrit or thrombocytes.

Any model intended to predict Dengue would need to be applied earlier in the illness to identify who must be closely monitored for plasma leakage (24). Although, in early phases, Dengue is often indistinguishable from other arboviral diseases (25). Using only clinical and socio-demographic data, as proposed in our work, may be a limitation in Dengue diagnosis. However, in remote areas that lack human and laboratory resources, the models can play an essential role in surveillance by identifying possible epidemics.

The OTHERS class performed a little better than the DENGUE class, but still showed weak performance. In this case, the great diversification may be the main cause, as this class includes all patients who were admitted with a suspected case of arbovirus, but were classified as inconclusive and discarded, so here the patients may include a wide variety of diseases.

The CHIKUNGUNYA class presented the best results in this work, principally in regards to the recall metric, with over 80%. These results show that, besides the difficulty, it is possible to make a good classification using only clinical and socio-demographic data. As such, our models can be used as a low-cost and rapid alternative, which would be useful in a resource-limited scenario (10). Note that arthralgia is considered a very common presentation in Chikungunya fever. Together with high fever, it has a specificity of 99.6% and a positive predictive value of 84.6% for infection classification. Since other arboviruses infections such as ZIKV, DENV and other alphaviruses also present with arthralgia, the epidemiological scenario must be considered before classifying every case as CHIKV infection based only on this symptom (26).

The tree-based models (Adaboost, RF, GBM and Xgboost) presented the best overall results. Tree-based models are generally the best models for problems that use tabular data. The MLP model presented the worst results, and a drop in performance was observed compared to training with grid search, thus indicating a possible underfitting.

The GBM model obtained the best results, and its attributes were analysed by the specialists and a new GBM model was designed. Despite obtaining slightly inferior results, it did have better interpretability for physicians and these results show that the GBM-SFA model is the most accurate model. On the order hand, the GBM-Specialist model is more interpretable and consequently would be more accepted for use by physicians. According to Ozaydin et al. (2021), “interpretability and accuracy may often have to be sacrificed for each other” ( 27). It is useless for a model to be precise if physicians cannot use it daily because they do not trust the model or do not understand the attributes used, for example. In this sense, the GBM-Specialist has a big advantage over the GBM-SFA model and, despite achieving lower performance, the difference in results is around 2% for each metric.

Although multi-class classification better represents real-world problems, most classification techniques are focused on binary classification (28). This happens, among other factors, due to the high complexity of training a model to be able to generalize more than two classes. This greater complexity contributes to multi-class models having lower performance when compared to binary models.

As future work, we plan to make an ensemble of two binary models, one trained to classify Dengue and another trained to classify Chikungunya. In this way, we can use more data from Dengue notifications, and we believe that with more data for training it is possible to improve the results of the DENGUE class.