Feature selection in wind power forecasting is a crucial step for improving model performance Meng et al. (2022). Correlation analysis methods like Pearson correlation coefficient can assist in selecting meteorological features to enhance forecasting accuracy (Liu et al., 2017). However, the Pearson correlation coefficient method overlooks the changing contributions of meteorological features to forecasts over time. The presence of the softmax function gives the attention mechanism excellent time selection capabilities. We introduce a feature attention mechanism to extract crucial features at different time points in real-time, while suppressing irrelevant features. This improves the adaptability and accuracy of the model. Figure 1 depicts the basic idea behind the feature attention method used in this investigation.

The feature attention mechanism first calculates the similarity between wind power and meteorological features at the same time point to obtain attention scores for each element. Taking time step t as an example, the calculation of its attention score is shown in Eq. 1. e t = v e ⁡ tanh w e u t + b e (1) where u t = [ u t ( 1 ) , u t ( 2 ) , u t ( 3 ) , u t ( 4 ) , u t ( 5 ) ] T represents the feature input vector at time step t, which includes five features for that moment: power, wind speed, wind direction, temperature, and air density; v _e and w _e is the weight. b _e is the bias term, and e _t encapsulates the attention score information for the five features at time step t.

Eq. 2 displays the probability distribution for the corresponding attention scores that are calculated using the softmax function. β t m = exp e t m ∑ i = 1 5 exp e t i (2) where β t m contains the importance degree corresponding to the m-th feature at the moment t. Finally, based on this importance degree and the input features, the feature attention output s _t is obtained, as shown in Eq. 3. s t = ∑ i = 1 5 β t i w s u t i (3)

The influence of historical data with different values and positions on the forecasting point varies. Attention mechanisms can capture these relationships. However, standard self-attention mechanisms lack control over the positional relationships in time series data. This results in assigning similar importance to historical data with different relative positions (Shih et al., 2019). To address these issues, this paper adopts the method proposed in reference (Yang et al., 2018) with some modifications. One of the modifications includes incorporating a Gaussian bias into the attention mechanism. This modification ensures that varying weights are assigned to data for each time steps. The Gaussian bias assigns different weights to attention scores for different positions, following a Gaussian distribution. The center position of the Gaussian function is automatically adjusted during parameter learning to focus on the region that is highly influenced by historical information for the current forecasting value.

The temporal attention mechanism is illustrated in Figure 2, with the input being the one-dimensional convolutional output sequence h ′ = [ h 1 ′ . ⋅ ⋅ ⋅ , h t − 1 ′ , h t ′ ] T . The calculation of attention scores in the temporal attention layer is described in Eq. 4. c = V c ⁡ tanh W c h ′ + b c (4) where V _c and W _c are weight matrices; b _c represents the bias term, and the attention score c = [ c 1 , ⋯ c t − 1 , c t ] T .

To achieve differential weight allocation across different time steps, Gaussian bias and attention scores are jointly input into the softmax function to compute attention probabilities α _t , as shown in Eq. 5. α t = s o f t max w α c t + G t (5) where w _α represents a weight factor; α _t represents the probability of the attention score at time step t, and G _t represents the Gaussian bias at time step t. G _t reflects the degree of closeness between the current moment and the central position moment, and its calculation method is as follows: G t = − t − Q t 2 2 σ 2 (6) Q t D = I ⋅ s i g m o i d q t z (7) q t = v q ⁡ tanh w q u t (8) z = v z ⁡ tanh w z K ̄ (9) where Q _t represents the center position at time step t, and its value is ultimately determined based on the parameter u _t , which is learned based on the value of t; σ is set to D 2 , where D is the window size for this mode, each mode has a separate window to define its window range, with a larger value indicating a longer sequence related to the current time step; I is a real number that ranges from 0 to the input sequence’s length; v _q , v _z , w _q and w _z are weight coefficients, and z is the scalar factor for selecting the window for this mode; K ̄ in the original reference represents the key-value relationship between semantics. Here, it is set to be the same as the kernel size k of the one-dimensional convolution. The underlying idea is that if a mode has a larger convolution kernel, its window will be larger. This allows different sequences to have different linear characteristics, aiding the model in capturing trend components of time series data.

Eq. 10 illustrates how the output of the temporal attention layer is finally calculated based on the probability α _i of the temporal attention scores. y t ′ = ∑ i = 1 t α i w y ′ h i ′ (10)

A single temporal attention mechanism is responsible for capturing the temporal weights of a single channel, while the multi-head temporal attention mechanism simply combines them. Applying one-dimensional convolutions with multiple diverse kernels to filter the GRU network’s output is necessary to guarantee that each temporal attention mechanism can extract distinct patterns of temporal information. When the length of the convolution kernel is k, the convolution formula is as shown in Eq. 11. h i ′ = R e L u ∑ l ∈ M H w l h i l + b l (11) where h i ′ represents the ith element of the feature map; R e L u ⋅ represents the activation function. M _H represents the spatial extent of the convolution kernel; w _l is the corresponding weight; h _il represents the lth element in the input data with i as the center; b _l is the bias of the convolution kernel. After the convolution output undergoes feature extraction through the temporal attention mechanism, the results are summed up with step-wise weighting, as shown in Eq. 12. p t = ∑ i = 1 H w p y i t ′ (12) where the projected output of the model at time step t is denoted by p _t ; The output of the ith temporal attention head at time step t is denoted by y i ( t ) ′ ; w _p corresponds to the weight associated with it. H represents the total number of attention heads.

Figure 3 shows the details of the implementation of the multi-head temporal attention mechanism.

Compared to the complex LSTM, the GRU has a simpler structure and higher computational efficiency. In this paper, the GRU network is chosen as the main component to construct the model. The feature selection and forgetting functions of GRU are implemented by only reset gates and update gates. GRU is relatively more efficient in terms of computational efficiency and number of parameters due to its simple structure. The update gate determines how much of the past information is retained, and the parameter values of the update gate are learned through training thus allowing the GRU unit to dynamically capture long-term dependencies in the sequence. The reset gate controls the inflow of historical information into the candidate hidden state and thus determines whether to disregard past information (van Heerden et al., 2022). Its internal structure is depicted in Figure 4.

Taking moment t as an example, the inputs to the gated loop unit are s _t and the hidden state h _t−1 from the previous moment. Inputs are processed to calculate the outputs of the update gate and reset gate, which are represented by Eqs 13, 14. r t = σ w s r s t + w h r h t − 1 (13) z t = σ w s z s t + w h z h t − 1 (14) where w _sr , w _hr , w _sz and w _hz represent weight terms; r _t is the output of the reset gate, and z _t is the output of the update gate; σ represents the sigmoid function.

The reset gate is used to determine the significance of the output from the previous time step. It combines this information with the current input to calculate the current hidden state, as shown in Eq. 15. h ̃ t = tanh w s h s t + w h h h t − 1 ⊗ r t (15) where w _sh and w _hh represent the weight terms for the importance at time step t and time step t − 1, respectively. h ̃ t signifies the hidden state at time step t, and ⊗ denotes the Hadamard product.

Eq. 16 is the final calculation used to determine the output of the GRU at the current time. h t = h ̃ t ⊗ z t + h t − 1 ⊗ 1 − z t (16)

According to Eq. 16, when the update gate output z _t approaches 0, the output of the GRU unit is mainly determined by the output of the previous time step. Conversely, when z _t approaches 1, it is primarily determined by the hidden state of the current time step (Liu et al., 2023).

Wind power is subject to weather conditions that are highly random and volatile. In order to decompose and predict wind power generation while considering climate features comprehensively and avoiding situations where historical information is insufficiently learned, such as forecasting lag, we establish a forecasting model that combines feature-based temporal dual attention mechanisms with GRU networks. The complete structure of the model is depicted in Figure 5.

The DAM-GRU model-based short-term wind power forecasting procedure is as follows:

1) Normalize the power, wind speed, temperature, and air density features using Eq. 17. u norm = u − u min u max − u min (17) where u _norm represents the normalized result. The text represents the values of the four mentioned feature sequences. The maximum and minimum values in the raw data are denoted by u _max and u _min, respectively. Normalize the wind direction feature using Eq. 18. u norm 3 = sin u 3 (18) where u norm ( 3 ) represents the normalized result of the wind direction feature, and u ⁽³⁾ represents the wind direction feature sequence.

2) The FAM assigns different weights to meteorological features based on different weather conditions. This allows it to dynamically extract key features in real time, improving the forecasting process.

3) Making initial predictions using a two-layer GRU model based on the extracted features.

4) One-dimensional convolutional models with H convolutional kernels of different sizes are used to apply convolutional filtering operations to the initial forecasting results of the GRU model. This is done to extract the timing patterns of wind power for different time series and reduce the complexity of individual timings.

5) In order to avoid forecasting lag, assign temporal feature weights to the H distinct periodic patterns independently using a multi-head temporal attention technique.

6) Combine the predictions of each sub-sequence by performing a weighted summation. Reverse the normalization process to obtain the final forecast result.

In order to statistically assess the accuracy of the wind power forecasting model, we compare RMSE, MAE, R ² SSE, MAPE, and RRMSE using the following formulas: R M S E = 1 n ∑ i = 1 n p ̂ i − p i 2 (19) M A E = 1 n ∑ i = 1 n p ̂ i − p i (20) R 2 = 1 − ∑ i = 1 n p ̂ i − p i 2 ∑ i = 1 n p ̂ i − p ̄ 2 (21) S S E = ∑ i = 1 n p ̂ i − p i 2 (22) M A P E = 1 n ∑ i = 1 n p ̂ i − p i p ̂ i × 100 % (23) R R M S E = 1 n ∑ i = 1 n p ̂ i − p i p ̂ i 2 (24) where p ̄ denotes the average value of the actual value sequence, n represents the number of forecast steps, p _i and p ̂ i represent the predicted and actual values for the ith sample point, respectively. Smaller RMSE, MAE SSE, MAPE, and RRMSE indicate better model performance, while an R ² closer to 1 indicates higher accuracy (Tian 2021; Tian et al. 2021).

The forecasting model employs feature sequences of length 30 steps as input, meaning that wind power for future n time steps is predicted using the previous 30 steps of features. The number of FAM heads in the model is equal to the number of input time steps. A double-layer GRU network is employed for the preliminary forecasting of wind power, with each layer containing 64 GRU units. The choice of attention heads is crucial, and in this case, parameter tuning experiments are conducted under 20 steps forecasting horizon. The number of attention heads, denoted as H, is tested with values of 2, 3, 4, 5, and 6, and the fluctuation of errors under different attention head numbers is shown in Table 1. The minimum error is achieved when H is set to 4. The convolutional kernels corresponding to the 4 attention heads have sizes of 3 × 1, 5 × 1, 7 × 1, and 9 × 1, with a stride of 1, the edge padding strategy is set to “same”, and the activation function is set to “ReLU”. The number of time steps in which the model outputs its forecasts is the number of output time steps for each TAM.

In this paper, three core components of the DAM-GRU model are the FAM, the GRU network, and the MHTAM. In order to emphasize the diversity of each time feature, a Gaussian bias is added to the multi-head time attention module. To evaluate the influence of each component of the DAM-GRU model on forecasting performance, this section compares the model with other models that use the same dataset and GRU network parameters. The DAM-GRU models compared include the GRU model, GRU-TAM model, FAM-GRU model, and GRU-MHTAM model. The GRU-TAM model combines a temporal attention mechanism with a single head within the GRU network. To ensure a fair comparison in the experiments, a 3 × 1 convolutional kernel is used to filter the initial forecasts made by the GRU. The FAM-GRU model combines the FAM with the GRU network, while the GRU-MHTAM model combines the GRU network with the multi-head temporal attention mechanism. The forecasting horizons are categorized into three levels: 20 steps (5-h), 40 steps (10-h), and 80 steps (20-h). Correspondingly, the lengths of the historical sequences are 30 steps, 60 steps, and 120 steps. This means that the model uses historical input sequences of 30, 60, and 120 steps to forecast power values for 20, 40, and 80 steps into the future, respectively.

Figure 6 show the comparison of forecast curves and box plots for five models at forecasting durations of 5-h, 10-h, and 20-h. All three sets of forecast curves show that the single GRU forecast model has a significant lag. This lag is attributed to its limited ability to efficiently extract highly correlated meteorological and time series features. The inclusion of the FAM and single-head temporal attention mechanism results in varying degrees of improvement, as evident in three box plots. The incorporation of the MHTAM leads to a significant enhancement in model performance. The proposed model in this paper demonstrates excellent feature extraction capabilities, making it adaptable to the rapid fluctuations in wind power. Overall, it demonstrates a significant decrease in lag and improved alignment with the observed values compared to the other models. Overall, the errors in model forecasts increase rapidly with increasing forecast length, and certain models have varying degrees; of outlier forecasts. This implies that the models require additional features to account for variations in the meteorological data as the forecast time horizon increases. The forecasting process becomes more complex and challenging due to various external factors that can impact the output of wind power.

Table 2 shows a comparison of the predictive performance for different attention mechanisms at 5-h, 10-h, and 20-h. From the table, it can be observed that the forecasting errors of all models increase rapidly as the forecasting steps increase, while the goodness-of-fit indicator, the coefficient of determination (R ²), decreases. This suggests that longer-term sequences exhibit weaker regularity and higher complexity, making it more challenging for models to extract features. The performance of the forecasting model can be improved by incorporating temporal and feature attention mechanisms. The model can more easily adjust to the intricacy of wind power sequences because of these attention mechanisms, which provide it the flexibility to focus on various attributes and time steps. Taking the 20-h forecast results as an example, the GRU-TAM model reduced the RMSE, MAE, SSE, MAPE, and RRMSE error metrics by 26.3%, 18.5%, 45.6%, 19.1%, and 24.6% respectively, and increased the R ² by 3.4% compared to the GRU model. On the other hand, the FAM-GRU model reduced the RMSE, MAE, SSE, MAPE, and RRMSE errors metrics by 35.8%, 32.4%, 58.8%, 25.2%, and 33.8% respectively, and increased the R ² by 4.3% compared to the GRU model. This demonstrates the necessity of adding attention mechanisms to help the GRU model extract features and temporal information. The GRU-MHTAM model, compared to the GRU-TAM model, reduces the RMSE, MAE, SSE, MAPE, and RRMSE error metrics by 53.0%, 56.4%, 78.9%, 53.7%, and 52.8% respectively, and increases R ² by 3.0%. This suggests that the MHTAM can be effective in dealing with time series features. In this experiment, the parts of the DAM-GRU model are split and tested for comparison, which proves the validity of the model design method from two perspectives: feature and time series, and demonstrates that the DAM-GRU model has good forecasting performance.

Table 3 shows a comparison of the time required for training and forecasting for the five models. The time required for training and prediction is not strictly increasing or decreasing due to complex factors such as GPU performance and initial parameters within the model, but generally shows some degree of regularity. From the table, it can be seen that the sum of training time for FAM-GRU and GRU-MHTAM is similar to the sum of training time for GRU and DAM-GRU, and the forecast time has the same pattern, which is consistent with the number of parameters of the model. As the number of forecast steps increases, the time required for training and forecasting may also increase, due to the fact that an increase in the number of forecast steps corresponds to an increase in the number of units in the input layer of the model.

In order to test whether the proposed DAM-GRU model has advanced prediction performance, we select CNN-GRU (Gao et al., 2023), VMD-CNN-GRU (Zhao et al. 2023), and MTTFA-GRU (Liu and Zhou, 2024) under the same data set algorithms as comparison models for the experiment. Among them, the number of input features of CNN-GRU and MTTFA-LSTM models is 5, the VMD-CNN-GRU model uses VMD decomposition to decompose the wind speed into four submodules, and then combines them with the wind power series using the CNN-GRU model to predict. To ensure the fairness of the experiments, the training batch size and iteration number of the comparison models are the same as the models in this paper, using the same settings.

At forecast horizons of 5 hours, 10 hours, and 20 hours, Figure 7 compare the suggested DAM-GRU model with three models. Box plots and forecast curves are compared in the figures. We find that the DAM-GRU model forecasts values that are quite similar to the observations. It demonstrates excellent predictive performance even during rapid changes in wind power over a short time frame. When combined with the accuracy metrics presented in Table 4, the DAM-GRU model outperforms the comparative models for forecasting horizons of 5 h, 10 h, and 20 h. Specifically, for a forecasting horizon of 5 h, the proposed model reduces RMSE, MAE, SSE, MAPE, and RRMSE forecasting errors by 6.0%, 18.9%, 11.7%, 20.6%, and 11.3% respectively, and increased the R ² by 0.8% compared to the MTTFA-LSTM model. For 10 and 20 h, The DAM-GRU model also outperforms the MTTFA-LSTM model to varying degrees. The DAM-GRU model’s performance is also superior to that of the CNN-GRU and VMD-CNN-GRU models. The validity of the DAM-GRU model was further confirmed.

To compare the training and forecasting time of the proposed DAM-GRU model with other mainstream and advanced GRU-based models. Table 5 shows that the proposed DAM-GRU model has a slight increase in training time and forecasting time compared to CNN-GRU, VMD-CNN-GRU, and MTTFA-LSTM models, but considering its obvious improvement in forecasting performance and that the time spent in actual online training is much less than the length of forecasting for each training, it can meet the technical requirements in practical applications.

In order to fully understand the difference between the forecasting performance of the DAM-GRU model and the traditional models, this section selects 4 traditional models including ARMA, RF, SVM and BP, and conducts comparative experiments using the same experimental strategy as in Section 4.3. The autocorrelation order of ARMA is set to 25, and the moving average order is set to 3. The number of decision trees of RF model is 80, and the minimum number of leaves is 5. SVM adopts Radical Basis Function (RBF) as the kernel function to build a regression model with multidimensional variables. The BP neural network has 6 units in the input layer, which are used to input 5 features and the forecasting results from the previous step, the hidden layer has 15 units for high dimensional mapping, and the output layer has one node for multi-step cyclic forecasting. Figure 8 shows a comparison of the forecast curves and box plots for each model when forecasting 20, 40 and 80 steps.

From Figure 8 it can be seen that the ARMA model is slightly stable at 5 h, but the error increases significantly as the prediction time increases, which may be less suitable for long-term forecasting. The three models, RF, SVM and BP, also show significant lags and do not fit the observed curves well at the peaks, where power changes more frequently. The DAM-GRU model proposed in this paper is able to accurately capture most of the large power variations and achieve accurate forecasts. This shows that the ideas and time-series feature processing methods specifically designed for wind power forecasting can make good use of historical data and capture the patterns embedded in longer time periods. Combining Tables 6, 7, it can be seen that although the proposed model has about twice the training and prediction time, its accuracy has improved significantly. As hardware performance improves, the training time for the model will be reduced even further in the future.