The gradient boosting decision is made to superimpose M sub-trees to achieve regression prediction: F ( x , w ) = ∑ m = 0 M α m h m ( x , w m ) = ∑ m = 0 M f m ( x , w m ) . (1)

In the formula shown previously, x represents the input sample, w m represents the model parameter, h represents the classification regression tree, and α signifies the weight of each tree. The core concept of GBDT algorithm is based on the weighted sum of multiple weak learners (Chen et al., 2015; Son et al., 2015; Sheridan et al., 2016; Rao et al., 2019; Wu et al., 2020; Yang et al., 2021a; Yang et al., 2021b; Shen et al., 2021). This study initializes several weak learners in the beginning: F 0 ( x ) = arg min c ∑ i = 1 N L ( y i , c ) . (2)

Then, building M trees, m = 1,2 , … , M :

1) For the sample i = 1,2 , … , N , the negative gradient corresponding to the number M tree is calculated by pseudo-residual:

The model of GBDT has many parameters, such as the number of base learners, the learning rate, the number of subsamples, and the maximum depth of each base learner (decision tree). Due to the limited number of defect data samples, the maximum depth of the tree should not be too deep.

Extreme gradient boosting algorithm is an ensemble learning algorithm based on gradient boosting. It calculates the final regression result by integrating multiple basic trees. It has advantages pertaining to high efficiency and accuracy in regression tasks. On the basis of GBDT, XgBoost introduces the loss function of the second derivative of the predicted results. It adds the tree model complexity into the objective function as a regular term. This can prevent over-fitting and improve the generalization performance of the model. In this study, the XgBoost prediction function is constructed as follows (Malhotra et al., 2015; Sikora and Al-Laymoun, 2015; Xu et al., 2015): y ^ i = ϕ ( x i ) = ∑ k = 1 K f k ( x i ) , f k ∈ F . (7)

Since the model is additive, the current prediction results need to be added to calculate in each iteration. L ( t ) = ∑ i = 1 n l ( y i , y ^ i t − 1 + f t ( x i ) ) + Ω ( f t ) . (8)

The overall objective function is: L ϕ = ∑ i l ( y ^ i , y i ) + ∑ k Ω ( f k ) . (9)

The Ω ( f ) is the regular term, which is expressed as: Ω ( f ) = γ T + 1 2 λ ‖ w ‖ 2 . .(10)

Taylor expansion of L ( t ) is: L ¯ t = ∑ i = 1 n [ g i f t ( x i ) + 1 2 h i f t 2 ( x i ) ] + γ T + 1 2 λ ∑ j = 1 T w j 2 = ∑ j = 1 T [ ( ∑ i ∈ I j g j ) w j + 1 2 ( ∑ i ∈ I j h j + λ ) w j 2 ] + γ T . (11)

Then, the ideal weight of leaf node j in round t should be w j ∗ = − ∑ i ∈ I j g i ∑ i ∈ I j h i + λ . (12)

The quality score of the tree in round t is marked by the following formula: L ¯ t ( q ) = − 1 2 ∑ j = 1 T ( ∑ i ∈ I j g i ) 2 ∑ i ∈ I j h i + λ + γ T . (13)

For all leaf nodes to be split in this round, I L and I R represent the set of assumed leaf nodes after splitting, so the loss reduction after splitting can be measured by the following formula illustrated as: L split = 1 2 [ ( ∑ i ∈ I L g i ) 2 ∑ i ∈ I L h i + λ + ( ∑ i ∈ I R g i ) 2 ∑ i ∈ I R h i + λ − ( ∑ i ∈ I g i ) 2 ∑ i ∈ I h i + λ ] − γ . (14)

Compared to GBDT, XgBoost algorithm is a further optimization design, which can reduce model variance through row sampling. Also, it reduces over-fitting through learning rate setting. Moreover, it controls the tree growth through early stopping to avoid over-fitting.

Figure 1 illustrates the unit structure of LSTM. At every moment t, the weight calculated by LSTM is linked back to itself. The input to the LSTM unit is the previous state h t − 1 and the current input x t . The function of storing and forgetting information is realized through four fully connected neurons, namely, f t , i t , c ˜ t , and o t . Specifically, the forgetting gate f t determines how much previous information is transmitted forward; input gate i t controls the input information level; the forgetting gate o t determines the output of this time step (Singh, 2017; Hu et al., 2018). The formula for calculation is described as follows:

1) Input the sequence value x t at time t and the hidden layer state h t − 1 at time t-1, and determine the discarded information through activation function. At this time, the output is as follows:

In the previously stated formula, f t is the result of the forgetting gate state, and W f and b f are the residual weight matrix and bias, respectively. σ is the activation function, usually the tanh or sigmoid function.

2) The input gate state formula is illustrated as follows:

In the aforementioned formula, i t is the result of the input gate state, and c ˜ t is the input unit state at time t. W i and W c are input gate weight matrix and input unit state weight matrix, repectively, and b i and b c are the corresponding input gate bias and input unit state bias, respectively. tanh is activation function, and ∘ stands for multiplying by elements.

3) Output information of LSTM is determined by the output gate and unit state as shown in the following equation:

In the formula, o t is the output gate state result, and W o and b o are the weight matrix and output bias, respectively.

Stacking fusion algorithm reduces the generalization error of the whole model and improves the classification accuracy of the model via building a two-layer learner. This makes the second-layer model to learn the classification results of the first-layer model. Among them, T basic classification models are first used in the first layer. After inputting the original data, a result with the same data size is outputted as the input of the second-layer network. The output of each basic learner is used as input when training the second-layer learner, and the function of the second-layer learner is to integrate the output of the basic learner.

In this study, the integrated feature scoring model is used to evaluate, and the total score is averaged to avoid the limitations of single feature selection and finally optimizes the effect of feature selection. With the help of classical Pearson correlation coefficient analysis and selecting high linear correlation attributes, we can roughly find out the relevant attribute categories that have a great impact on material defects. The Pearson correlation coefficient formula is shown in Eq. 21 and Eq. 22. Through this formula, Pearson correlation coefficient can be obtained by dividing the covariance by the standard deviation of two related variables, which is used to make up for the weak performance of the covariance value in the correlation degree of random variables.. cov ( x , y ) = ∑ n i = 1 ( x i − x ) ( y i − y ) n − 1 ; (21) P e a r s o n = c o r r ( x , y ) = cov ( x , y ) σ x σ y = E [ ( x − μ x ) ( y − μ y ) ] σ x σ y . (22)

The range of the Pearson coefficient is [ − 1,1 ] . The larger the absolute value is, the more linearly related the two random variables are. Pearson = 1 means that the random variables are completely positively correlated, Pearson = - 1 means that the random variables are completely negatively correlated, and Pearson = 0 means that there is almost no linear correlation between the two variables [Feng et al., 2019; Li et al., 2021a; Li et al., 2021b].

In order to comprehensively evaluate the validity and accuracy of the proposed method in power grid material forecasting, the following two evaluation indexes are selected:

The value of the root mean square error (RMSE) between the real value of the test set and the predicted value of the model is expressed as follows: R M S E = ∑ i = 1 T ( x ˜ i − x i ) 2 T . (23)

The average relative error between the real value and the predicted value, E r r , is expressed as: E r r = 1 T ∑ i = 1 T | x ˜ i − x i x i | × 100 % . (24)

In this study, the prediction results of emergency defect materials are compared and analyzed, taking overhead wire as an example.

The number of sub-trees of XgBoost is set at 300, and the learning rate is kept at 0.04, while the penalty factors γ and λ of tree model complexity are taken as 0.01 and 0.9, respectively. Furthermore, the number of sub-trees of GBDT too is set at 300 along with the learning rate at 0.04. The network layers of four kinds of LSTM networks (LSTM-1, LSTM-2, LSTM-3, and LSTM-4) are set at 3, 3, 4, and 4, respectively; the number of corresponding neurons is set to 128, 128, 256, and 256, respectively, and the learning rate is set at 0.01.

Table 1 shows the prediction results of different base models, multi-model fusion, and multi-model fusion with feature optimization for emergency materials. The results of the basic model are obtained directly from their own network without the second layer fusion of the proposed network.

The results of the evaluation index reveal that a single model cannot achieve the best effects. After using the multi-model fusion algorithm, it can make full use and give full play to the prediction advantages of each basic model in one aspect. Moreover, it improves the overall prediction accuracy of the algorithm. It also avoids the phenomenon of over-fitting along with enhanced stability of prediction. On the basis of multi-model fusion, this study also uses Pearson coefficient to optimize the input parameters and selects the most important characteristic parameters as the input of the network. Therefore, after the feature optimization of the original input parameters, the main influencing feature parameters can be selected. Following this, the feature parameters of irrelevant factors can be eliminated, so that the subsequent learning model can better fit the historical data. Therefore, based on the aforementioned observations, it can be concluded that the multi-model fusion algorithm of feature optimization proposed in this study has high accuracy in the prediction of power grid materials.