This section presents the workflow of our proposed system for predicting COVID-19 infections along with the steps involved. It explains the method used to select the most accurate model for prediction based on the infection’s trend. The use of a systematic workflow for the problem of infection prediction is the most important for the accurate infection prediction for a given country. Figure 1A shows the seven stages involved in the proposed system. In the following, we explain each stage in detail.

In this section, the implementation of the real-time system is discussed. The suggested implemented diagram is shown in Figure 2. The infection data is collected from different data sources D_src such as the Ministry of Health, Hospitals, and public health agencies (for example WHO). The infection data Inf is stored using the data storage component. The raw infection data is stored as a data frame df_infis fed as an input to the data transformation component. The preprocessed data frame d⁢f′i⁢n⁢f is again stored. The transformed data is constantly updated in the storage in real-time using a data update feedback loop. The preprocessed data is then divided into training d⁢fi⁢n⁢ft⁢r and validation d⁢fi⁢n⁢fv⁢d datasets. A model is selected by the intelligent agent based on the temporal distribution of the infection data evolution. The selected model f(inf)is developed usingd⁢fi⁢n⁢ft⁢r. The performance of the model is evaluated usingd⁢fi⁢n⁢fv⁢d. The model development is a feedback control process where the model is tuned using hyperparameter tuning unless the desired performance is obtained. The infection prediction error e_infobtained from the evaluation is fed back to tune the hyperparameters. The tuned model f*(inf) is deployed for predicting infections accurately. The deployed model is updated in real-time using the model feedback loop when the infection data is updated. The healthcare organizations and the government then use the deployed model to predict the infections. This is by providing the input arguments, country for which the prediction is required, and the duration of prediction C, t. The number of infections for the prediction period Inf_t is sent to the healthcare organizations and the government.

To evaluate the performance of our proposed system, we developed the prediction models for fifteen countries based on the literature (Table 1). We used the Johns Hopkins COVID-19 dataset that is updated daily.⁶ Table 2 presents the countries for which the prediction models are developed, the features of the dataset, data update frequency, and the period for which the COVID-19 infections data are extracted for the countries under study. The dataset has no outliers and missing values. We used the number of confirmed cases for each country to develop the models. Figure 3 shows the infection trend for the considered countries. As shown in the figure, the distribution of the infection growth for each country is different. In this paper, we use country-level data for the evaluation as the dataset does not include city-level or state-level data for the countries under study. However, the system can be used for city-level or state-level infection data as well.

To predict the COVID-19 infections for the countries under study, we used our proposed system that selected the most accurate machine/deep learning model based on the temporal distribution of the infection evolution for a country (31) as stated in Figure 1. For each country under study, we compared the performance of the model selected using our proposed system with the outperforming model(s) in the literature for that country (Table 1). Table 3 presents the selected model and the models used for the comparison for each country. The description and the parameters for the models are listed in Table 4.

To develop the prediction models, we create a separate dataset for each considered country. We use 70% of the dataset (i.e., 22/01/2020—06/06/2021) for training (develop) the model and 30% of the dataset (i.e., 07/06/2021–08/01/2022) for validating the developed model. We first developed a model for each country using the training dataset for that country. We then validated the developed model by predicting the number of infections for the validation period, i.e., 07/06/2021–08/01/2022, and comparing the predicted values with the actual ones. In addition, we developed the outperforming model(s) for each country under study based on the literature (Table 1) and predicted the infections using the developed model(s). We evaluate the performance of the models in terms of RMSE and MAPE that are computed using Equations (1) and (2), respectively.

where n is the total number of days for which the infections are predicted

To tune the hyperparameters for the considered models, we implement each model with varying parameters’ values and select the values that result in the least MAPE. In particular, to obtain the values of α and β parameters for HLT model, we implement the model with varying values of the parameters between [0, 1] at an interval of 0.1, i.e., (α = 0, β = 0), (α = 0, β = 0.1), …, (α = 0.2, β = 0), (α = 0.2, β = 0.1),… (α = 1, β = 1). The combination of values that return the minimum MAPE is selected. For QT, we implement the model for varying degrees of polynomial between [1, 10] and selected the degree resulting in the least MAPE value. To obtain the values of α, β, Ø parameters for the DT model, we implement the model with varying values of the parameters between [0, 1] at an interval of 0.1 and selected the combination of values that return the minimum MAPE. To obtain the values of input size, number of neurons, epochs, activation function, and optimizer for LSTM, Bi-LSTM, and Bi-GRU models, we first determine the values of input size, number of neurons, and epochs by brute-force method while using Rectified Linear Unit (ReLU) activation function and Adaptive Movement Estimation (Adam) optimizer. We then vary the activation function and optimizer by keeping other parameters constant at their optimal values. The input sizes of 10, 50, 100, 200, and 250 are considered for the experiments. The different values used for epochs are 100, 200, 300, 400, and 500. However, for Italy, 1500 epochs are used as the model did not converge with 500 epochs. The number of neurons is varied from 100 to 1,000 at an interval of 100. The different activation functions used are ReLU, Softplus, Softmax, Softsign, Scaled Exponential Linear Unit (SELU), Linear, Hard_sigmoid, Sigmoid, Hyperbolic Tangent (Tanh), and Exponential Linear Unit (ELU). The optimizers used for tuning are Adam, Adadelta, Adaptive Gradient (AdaGrad), Adamax, Nesterov-accelerated Adaptive Moment Estimation (Nadam), Stochastic Gradient Descent (SGD), and Root Mean Square Propagation (RMSprop). The Mean Squared Error (MSE) loss function is used for LSTM, Bi-LSTM, and Bi-GRU models. To yield parameters’ values for the ARIMA and SutteARIMA models, we first check the stationarity of the infection data and determine the value of d. This is by performing the statistical augmented Dickey-Fuller (ADF) test (33, 46) that checks the null hypothesis that the data is non-stationary and returns a probability score (p-value). A p-value < 0.05 indicates that the time series is stationary. If the p-value ≥ 0.05 (non-stationary time series), then the time series is differenced and the ADF test is performed again. This is repeated until the time series becomes stationary. The value of d is then equal to the number of times the series is differenced. After determining the value of d, we plot the Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots for the differenced time series to determine the values of q and p, respectively. The number of lags for which the ACF is outside the significant threshold represents the value of the parameter “q” value and the number of lags for which the PACF is outside the significant threshold represents the value of “p.”

Figure 4 shows the MAPE obtained by the HLT models, for different values of α and β, when developed for the infection data of China, France, Germany, Italy, and Malaysia. It shows that the minimum MAPE is obtained for (α, β) values of (0.1, 1.0), (0.3, 0.9), (1.0, 0.1), (1.0, 0.1), and (0.1, 0.4) for China, France, Germany, Italy, and Malaysia, respectively. We use these values to develop the prediction model for the corresponding countries. Figure 5 shows the MAPE obtained by the DT model, for different values of α and β, when developed for the infection data in Brazil, India, and Saudi Arabia. It shows that the minimum MAPE is obtained for (α, β) values of (1.0, 0.2), (1.0, 0.1), and (0.5, 0.1) for Brazil, India, and Saudi Arabia, respectively. We use these values to develop the prediction model. Figure 6 shows the training and validation losses over epochs for LSTM, Bi-LSTM, and Bi-GRU models for China, Germany, Italy, Australia, Iran, Russia, Spain, the United Kingdom, the United States, Israel, Brazil, and India. As shown in the figure, both training and validation losses converge, indicating a good fit. However, for Australia (Figure 6D), there is a gap between the training and validation losses indicating unrepresentative training dataset. This is because the number of infections for Australia increased rapidly during the validation period, as shown in Figure 3, which is not captured by the model develop using the training dataset. For the ARIMA model, we first perform the ADF test to check the stationarity of time series data for France, Italy, Malaysia, Spain, the United States, and Saudi Arabia. The p-values obtained for Malaysia, Spain, the United States, and Saudi Arabia after the second-order are 0.000000, 0.000000, 0.000092, and 0.000117, respectively. The p-values < 0.05 for these countries indicate that the time series becomes stationary after second-order differencing. Consequently, the value of d is set to 2 for these countries. For France and Italy, p-values < 0.05, i.e., 0.003894 and 0.048181, respectively, are obtained after first order differencing. However, the ACF plots for the first ordered differenced infection data of France and Italy do not converge to zero. Consequently, we differenced the time series for these countries one more time and select d = 2 for France and Italy after obtaining a p-value of 0.000000 and 0.001730, respectively. Figure 7 shows the ACF and PACF plots for the stationary infection data, i.e., after second-order differencing, for France, Italy, Malaysia, Spain, the United States, and Saudi Arabia. As depicted in Figure 7A, 1 lag value is outside the significant threshold in the ACF plot for France indicating q = 1. Moreover, 10 values in the PACF plot are outside the threshold indicating p = 10. Similarly, (p, q) values for Italy, Malaysia, Spain, the United States, and Saudi Arabia are (5, 7), (5, 2), (6, 8), (9, 1), and (3, 1) as shown in Figures 7B–F), respectively. Table 5 shows the optimal values of parameters for the developed models.

Figure 8A shows COVID-19 confirmed cases for the training and validation datasets for China. In addition, it indicates the number of infections forecasted by the HLT model selected using the proposed automated AI system and the Bi-LSTM model from the literature (25). It shows that HLT model predicts the infections with more accuracy compared to Bi-LSTM. This is because HLT fits well the exponential + linear infection trend for China. The (MAPE, RMSE) values using HLT and Bi-LSTM models for China are (1.29, 1934.36) and (11.39, 13331.86), respectively. Figure 8B shows the predicted infections for France using the proposed automated AI-selected HLT model and state-of-the-art ARIMA model (27). It shows that HLT outperforms ARIMA. As depicted in Figure 8B, HLT model predicts with lower error for the validation period where the infection’s trend is linear than where the trend is exponential. The prediction error for HLT increases as the infection grows exponentially toward the end of the validation period, which is not captured by the model. The (MAPE, RMSE) values using HLT and ARIMA models for France are (3.87, 702931.85) and (9.39, 1155417.17), respectively. The prediction for Germany using automated AI-selected HLT and state-of-the-art Bi-LSTM (25) is shown in Figure 8C. HLT outperforms Bi-LSTM as it can capture the exponential + linear infection trend for Germany. However, similar to Figure 8B, the prediction error by HLT for Germany (Figure 8C) increases when the validation infection data exhibits an exponential trend. The (MAPE, RMSE) values using HLT and Bi-LSTM models for Germany are (9.37, 967916.97) and (28.01, 1321353.74), respectively. Figure 8D shows COVID-19 prediction for Italy using automated AI-selected HLT and state-of-the-art ARIMA (24, 27) and Bi-LSTM (25) models. The HLT model outperforms ARIMA and Bi-LSTM models. The (MAPE, RMSE) values using HLT, ARIMA, and Bi-LSTM models for Italy are (2.84, 389747.98), (6.56, 581053.16), and (12.41, 837410.43), respectively. The prediction results for Malaysia using our automated AI-selected HLT model and state-of-the-art ARIMA model (28) are presented in Figure 8E. HLT captures the infection trend for Malaysia and outperforms ARIMA in predicting COVID-19 infections. The (MAPE, RMSE) values using HLT and ARIMA models for Malaysia are (16.37, 412523.95) and (23.23, 617834.31), respectively.

Figure 8F shows the prediction results for Australia using automated AI-selected QT and state-of-the-art LSTM (26). The (MAPE, RMSE) values using QT and LSTM models for Australia are (20.64, 80417.79), and (68.60, 181145.56), respectively. Figure 8G shows the prediction results for Iran using automated AI-selected QT and Bi-GRU (26). The (MAPE, RMSE) values using QT and Bi-GRU models for Iran are (8.54, 579794.14) and (31.90, 2086139.84), respectively. Figure 8H shows COVID-19 predictions for Russia using automated AI-selected QT and Bi-LSTM (25). It depicts that QT outperforms Bi-LSTM as it can capture the polynomial trend of the infection data in Russia. The (MAPE, RMSE) values using QT and Bi-LSTM models for Russia are (12.87, 941065.72) and (23.58, 2536117.98), respectively. Figure 8I shows the prediction results for Spain using automated AI-selected QT and state-of-the-art ARIMA (27), SutteARIMA (23), and Bi-LSTM (25). The (MAPE, RMSE) values using QT, ARIMA, SutteARIMA, and Bi-LSTM models for Spain are (5.77, 497155.75), (13.26, 825509.28), (56.48, 2804433.84), and (16.48, 1047913.19), respectively. Figure 8J shows the prediction results for the United Kingdom using automated AI-selected QT and Bi-LSTM (25). The (MAPE, RMSE) values using QT and Bi-LSTM models for the United Kingdom are (16.57, 1167306.58) and (27.40, 3450595.03), respectively. Figure 8K shows the COVID19 infection prediction for the United States using LT, ARIMA (24), and Bi-LSTM (25). The (MAPE, RMSE) values using LT, ARIMA, and Bi-LSTM models for the United States are (3.79, 2197376.04), (15.5, 9450564.22), and (10.99, 6337067.40) respectively. Figure 8L shows the prediction results for Israel using automated selected LT and Bi-LSTM (25). The (MAPE, RMSE) values using LT and Bi-LSTM models for Israel are (9.06, 119886.19) and (20.91, 335433.23), respectively. Figures 8M,N) show the prediction results for Brazil and India, respectively, using automated AI-selected DT models and Bi-LSTM models (25). They show that DT outperforms Bi-LSTM for both Brazil and India as it can accurately DT capture the exponential + damping trend of infection growth. The (MAPE, RMSE) values using DT and Bi-LSTM models for Brazil are (0.73, 175627.67) and (14.02, 3313775.77), respectively. The (MAPE, RMSE) values using DT and Bi-LSTM models for India are (4.79, 1732187.64) and (36.89, 12906730.59), respectively. Figure 8O shows the prediction results for Saudi Arabia using automated AI selected DT and ARIMA (29). The (MAPE, RMSE) values using DT and ARIMA models for Saudi Arabia are (1.54, 9909.39) and (6.37, 47768.10), respectively. Figure 9 show the MAPE and RMSE obtained by the model selected using the proposed system and state-of-the-art approaches for each country under study. It shows that the selected models outperform the approaches in the literature for each country. In summary, the proposed system predicts COVID-19 infections with an average MAPE and RMSE of 7.87 and 665052.14, respectively. The average MAPE values for state-of-the-art Bi-LSTM, ARIMA, LSTM, Bi-GRU, and SutteARIMA models are 20.21, 12.38, 68.60, 31.90, and 56.48, respectively, whereas the average RMSE values are 3209972.92, 2113024.38, 181145.57, 2086139.84, and 2804433.85, respectively.