The uncertainty of wind power output is primarily influenced by wind speed. To more accurately simulate the actual variations in wind speed, it can be represented using probability distributions such as the Weibull distribution, Gaussian distribution, and Pearson distribution. Historical data indicates that the actual wind speed aligns most closely with the Weibull distribution’s probability density function. Therefore, this paper employs the Weibull distribution function to characterize the wind speed and establish a probabilistic representation of the uncertainty between the wind turbine’s output active power and wind speed. The wind speed probability density function of the Weibull distribution, denoted as f(v), and the cumulative distribution function of the Weibull distribution, denoted as F(v), as shown in Equations 1, 2: f v = K C v C K − 1 exp − v C K (1) F v = 1 − exp − v C K (2)

Where v is the wind speed; K is the shape parameter of the Weibull distribution; C is the scale parameter of the Weibull distribution.

The characteristic curve of wind power output defines the relationship between wind power output and wind speed, where the intensity of wind speed directly influences the magnitude of the output. The relationship between wind power and wind speed can be described by a linear function, quadratic function, or cubic function, leading to distinct wind turbine power curves. Taking into account the actual statistical wind power data, wind power output is typically modeled using a cubic segmented function, which can be expressed as Equation 3: P W = 0 v < v in or  v > v out P r v 3 − v in 3 v r 3 − v in 3 v in ≤ v < v r P r v r ≤ v ≤ v out (3)

Where v _r, v _in, v _out are the rated wind speed, cut-in wind speed and cut-out wind speed of the wind farm turbine respectively; P _r is the rated power of the turbine.

Apart from temporal uncertainty, wind power output exhibits spatial correlation as well. Due to the close proximity of various wind farms within the same region and their placement in similar wind speed bands, a robust correlation exists between the outputs of different wind farms, consequently impacting the overall uncertainty of wind power. Hence, this section considers the spatial correlation among distinct wind farms and employs the Nataf inverse transformation principle to generate wind turbine output uncertainty data with predetermined correlation coefficients.

The theory of Nataf transform can transform random distribution variables with correlation into standard normal distribution variables that are independent of each other. The Nataf inverse transform serves as the reverse procedure to the Nataf transform, allowing the generation of distribution variables with desired correlation coefficients using independent standard normal distribution variables. This process facilitates the sampling of a significant amount of specified sample data.

Let the vector P W . i   i = 1 , 2 , … , n represent the active outputs of n Weibull-distributed wind farms in the original correlation variable space. Similarly, let the vector z i   i = 1 , 2 , … , n denote the n standard normally distributed random variables in the correlation standard normal space. Subsequently, assume that the linear correlation coefficient matrices for Z and P _W are denoted by ρ ₀ and ρ, respectively. Here, ρ is a predetermined value, and the relationship equation between the elements of the ρ ₀ and ρ matrices is given as: ρ 0 i j = R i j ρ i j (4) R i j = 1.063 − 0.004 ρ i j − 0.200 γ i + γ j − 0.001 ρ i j 2 + 0.337 γ i 2 + γ j 2 − 0.007 γ i γ j (5)

Where γ _i and γ _j represent the computational parameters of the random variables P _i and P _j , respectively. The expressions for these parameters are given as follows Equation 6: γ i = σ i / μ i γ j = σ j / μ j (6)

The positive definite symmetric matrix of correlation coefficients ρ ₀ can be obtained through Equations 4, 5, and it can be decomposed into the lower triangular matrix B using the following expression Equation 7: ρ 0 = B B T (7)

A standard normal distribution vector Z with specified correlation coefficients can be generated from the pre-obtained independent standard normal distribution vector X. The transformation is shown as Equation 8: Z = B X (8)

Based on the equal probability transformation criterion, the standard normal distribution space with correlation is converted into correlated input vectors, i.e., wind power output variables that follow the Weibull distribution. The output power of each wind power node is given by Equation 9: P W . i = F i − 1 Φ z i (9)

Where P _{W. i} represents the correlated active power output of wind power node i; F i − 1 ⋅ is the inverse cumulative distribution function of the active power output of wind power node i; Φ(z _i ) denotes the cumulative distribution function of z _i .

The optimization strategy presented in this paper encompasses various novel controllable load types like electric vehicles, energy storage systems, commercial buildings, 5G base stations, and distributed photovoltaics. These loads can be directly enlisted by the emergency control system for urgent load shedding and contribute to the emergency frequency control of the power system. Unlike traditional methods that directly cut the load line during emergency frequency control, these controllable loads have a reduced impact on users when temporarily removed, resulting in lower load shedding costs. Furthermore, the power of these controllable loads can be precisely regulated by power electronic devices, enabling more flexible engagement in the power system’s emergency frequency control. The diverse characteristics of controllable loads introduce a complex influence on emergency frequency control, posing challenges in integrating them for considerations such as control continuity and data reliability. Consequently, the load side fluctuation range in modern power systems has expanded, while the time scale has diminished. This, in turn, has led to an escalation in power demand uncertainty, necessitating the characterization of load power uncertainty.

The probability of load power uncertainty is modeled using a normal distribution, which is expressed through a probability density function, as shown in Equation 10: f P L = 1 2 π σ P L exp − P L − μ P L 2 2 σ P L 2 f Q L = 1 2 π σ Q L exp ⁡ − Q L − μ Q L 2 2 σ Q L 2 (10)

Where P _L and Q _L represent the active and reactive power of the load, respectively; μ _PL and μ _QL denote the expected values of the active and reactive power of the load, respectively; σ _PL and σ _QL indicate the standard deviation of the active and reactive power of the load, respectively.

Additionally, the presence of various new controllable loads on the load side, such as electric vehicles and energy storage, introduces variability and diversity in load characteristics. The complexity of these controllable load components further contributes to the uncertainty of overall load characteristics. Determining the controllable load characteristics directly becomes infeasible when the power system’s operating state changes, necessitating the expression of uncertainty through a probability distribution. Consequently, a novel static load model should be established utilizing frequency and voltage indices that adhere to the probability distribution, as Equation 11. P L . new ′ = P L U / U N k p u . new 1 + k p f . new f − f N Q L . new ′ = Q L U / U N k q u . new 1 + k q f . new f − f N (11)

Where k _{pu. new} and k _{qu. new} represent voltage indices of active and reactive power of the new controllable loads, respectively; k _{pf. new} and k _{qf. new} denote frequency indices of active and reactive power of the loads, respectively.

These parameters, k _{pu. new}, k _{qu. new}, k _{pf. new} and k _{qf. new}, are subject to uncertainty and are characterized by probability distributions that follow a normal distribution.

In summary, considering the uncertainty of load size, which is represented by P _L and Q _L that conform to normal distribution, and considering the uncertainty of load characteristics, which is represented by P ^’ _L.new and Q ^’ _L.new that contain time-varying load coefficients, a power demand uncertainty model that integrally considers fluctuations in load quantity and fluctuations in load characteristics is thus established.

Power system emergency frequency stabilization is closely related to generator active power, load power, system frequency, and the rate of frequency change. Considering the dual source-load uncertainty in power-side active output and demand-side active load, it is necessary to incorporate all generator active output and load node power with uncertainty into the state space, defining the state space s _t as Equation 12: s t = s 1 t ∪ s 2 t ∪ s 3 t ∪ s 4 t s 1 t = f 1 t f 2 t ⋯ f m t s 2 t = d f / d t 1 t d f / d t 2 t ⋯ d f / d t m t s 3 t = P e . 1 t P e . 2 t ⋯ P e . m t s 4 t = P l . 1 t P l . 2 t ⋯ P l . n t (12)

Where f _i ^t is the frequency of generator node i at moment t; (df/dt) _i ^t is the frequency change rate of generator node i at moment t; P _{e. i} ^t is the electromagnetic power of generator node i at moment t; P _{l. j} ^t is the active load of load node j at moment t.

The control action of each controllable load at moment t should be to reduce a part of the total controllable load at that node. Due to the uncertainty of load demand power, the total controllable load needs to be updated in real time. However, for uniformity of the control action, the action space must be fixed. Therefore, the action space is set as the proportion of the controllable load removed at each node. The actual load reduction is the value of the action at each node multiplied by the total controllable load at that node. Consequently, each controllable load action is defined as a continuous value within [-1, 0], and the total action space is shown as Equation 13: a t = Δ P 1 t Δ P 2 t ⋯ Δ P n t (13)

Where ΔP _m ^t is the load removal of controllable load node m at time t and ΔP _m ^t ∈[−1,0]; n is the number of controllable load nodes.

The goal of the emergency frequency control problem is to restore the frequency to within the stabilization range quickly while minimizing load shedding. For source-load dual uncertainty power systems, the effectiveness of emergency frequency control is primarily evaluated in terms of frequency deviation and load shedding amount.

Therefore, the reward function consists of three parts: 1) the average value of steady-state frequency deviation over a specific time period at the end of the simulation; 2) a penalty term calculated based on controllable load importance and load shedding; and 3) a penalty term for exceeding the lowest point of the system’s dynamic frequency. The expression is shown as Equation 14: r t = λ 1 Δ f T tem − λ 2 ∑ j = 1 n C j P sl . j − H 1 H 1 = − 100 , if  f min < f min ⁡ . set 0 , therwise (14)

Where T _tem is a certain time period before the end of the simulation process; Δf _Ttem is the average value of the deviation of the center of frequency inertia during T _tem; C _j is the importance index of load node j; P _{sl. j} is the amount of load shedding at node j; H ₁ is the penalty for the system’s center of frequency inertia when the minimum value is less than the integrating value; λ ₁ and λ ₂ are coefficients for each part of the reward function.

The SAC algorithm belongs to the deep reinforcement learning algorithms based on the value function, which incorporates a mechanism that encourages exploration through action strategy entropy values. This enhances the algorithm’s robustness compared to other strategy gradient-based DRL algorithms like PPO, A3C, and DDPG. The entropy value, defined as the expectation of information quantity, quantifies the uncertainty of a variable. It increases with the uncertainty of an event and can be quantified by the event’s probability. The entropy value is defined as Equation 15: H X = − ∑ x i ∈ X l x i ln ⁡ l x i (15)

Where H(X) is the entropy value; l (x _i ) is the event probability.

The DRL algorithm should continuously explore the interaction environment to accumulate experience and avoid selecting too many actions solely based on immediate rewards, as this may lead to convergence on local optimal solutions. The SAC algorithm considers the maximum entropy value of actions. If the entropy value decreases due to repeated selection of a certain action, the maximum entropy mechanism encourages the agent to explore other actions, thus broadening the exploration range and increasing the algorithm’s robustness.

In other deep reinforcement learning algorithms with stochastic policies, the objective of model learning is clear: to derive an optimal action policy that maximizes the expected cumulative reward through straightforward training. The optimal policy expression is shown as Equation 16: π = argmax π E s t , a t ∼ ρ π ∑ t r s t , a t (16)

The SAC algorithm necessitates maximizing the entropy value of the output action to enhance exploration capability. In other words, an additional term regarding the entropy value is incorporated into the policy expression, resulting in the expression of the improved optimal policy as shown in Equation 17: π = argmax π E s t , a t ∼ P π ∑ t r s t , a t ⏟ reward + α H π ⋅ ∣ s t ⏟ entropy (17)

Where E (s _t , a _t ) denotes the expectation function; π represents the strategy; s _t and a _t signify the state space and action space at moment t; r (s _t , a _t ) denotes the reward function at moment t (s _t , a _t )∼P _π signifies the trajectory of state-action under strategy π; + is the automatic entropy temperature parameter, which adjusts the entropy value affecting the degree of rewards; and H (π(⋅|s _t )) signifies the entropy of the output action of the strategyπ under the state s _t , as expressed below in Equation 18: H π ⋅ ∣ s t = − ∑ π ⋅ ∣ s t log π ⋅ ∣ s t = − ∫ a t P π a t ∣ s t ln ⁡ P π a t ∣ s t d a t (18)

Where P (π (a _t |s _t )) denotes the probability that the action value at the time of t is a _t .

In the SAC algorithm for strategy value evaluation, the expression for updating the strategy using the Bellman operator is expressed as Equation 19: Q π s t , a t = r t + E ∑ t = 1 ∞ γ t r s t , a t − α ⁡ log ⁡ π a t ∣ s t (19)

Where γ denotes the discount factor at the time of strategy update.

The optimal policy can be continuously learned and refined through policy iteration, comprising two steps: soft policy evaluation and soft policy improvement. Firstly, in the strategy evaluation step, the soft value update function of a given strategy π can be obtained using the soft Bellman operator, as shown in Equation 20: T π Q π s , a = r + γ E s ′ Q π s ′ , a ′ − α ⁡ log ⁡ π a ′ ∣ s ′ (20)

The SAC algorithm belongs to the Actor-Critic class of algorithms, where the Actor is employed for policy modeling and the Critic for Q-value function modeling. Different deep neural networks are utilized to fit the Q-value function and the policy function, respectively, as shown in Equation 21: J Q θ = E 1 2 ( Q s t , a t − r s t , a t + γ V θ ¯ s t + 1 2 (21)

Where θ denotes the parameters of the policy network; V θ ¯ represents the updated value function value.

Both networks are optimized using independent gradients ∇ ^ θ J Q θ , as expressed in Equation 22: ∇ ^ θ J Q θ = ∇ θ Q θ s t , a t ⋅ Δ Q θ (22)

Where the expression of ΔQ _θ is expressed as Equation 23: Δ Q θ = Q θ s t , a t − r s t , a t + γ Q θ ¯ s t + 1 , a t + 1 − α ⁡ log π ϕ a t + 1 ∣ s t + 1 (23)

The outputs of the policy network are the mean and standard deviation values following a Gaussian distribution. The network with the smaller Q value is selected to reduce bias in updating the parameters of the policy network. The approximate gradient of the parameter update is expressed as Equation 24: ∇ ^ ϕ J π ϕ = ∇ ϕ α ⁡ log π ϕ a t ∣ s t + ∇ a t α ⁡ log π ϕ a t ∣ s t − ∇ a t Q s t , a t ∇ ϕ f ϕ ε t ; s t (24)

At the same time, the action entropy value is also updated in the policy network, making it crucial to choose the appropriate temperature parameter, α. As the reward value varies during the training process, fixing the temperature coefficient reduces the stability of model training. Therefore, the temperature coefficient α is generally updated automatically by minimizing J (α), as expressed in Equation 25: J α = E a t − π i − α ⁡ log π t a t ∣ s t − α M (25)

Where M represents the dimension of the action matrix, specifically denoted as M = dim(a).

The SAC algorithm for deep reinforcement learning comprises four crucial components: the experience replay buffer, the automatic entropy parameter, the policy network, and the value network. The experience replay buffer stores historical exploration experience, while the automatic entropy parameter stabilizes and adjusts the exploration strategy. The policy network is responsible for action selection, and the value network estimates state-action values. The overall structure of the algorithm is depicted in Figure 2.

When utilizing the SAC algorithm to optimize the emergency frequency control strategy, each iterative training process can be summarized into three main steps: firstly, collecting and inputting the operating state data of the power system after the fault into the SAC model; then, the SAC model selects the emergency frequency control action based on the state data; finally, executing the control action on the power system simulation environment to achieve the objective. Additionally, due to the uncertain nature of source-load power systems, it is necessary to incorporate an uncertainty model for wind power output and load demand in each interaction process. The overall process of emergency frequency control for a source-load dual uncertainty system based on the SAC algorithm is illustrated in Figure 3.

Prior to model training, the simulation environment and SAC model parameters are initialized. The power system load factor is randomly initialized, and the model incorporates uncertainty in wind power output and load demand. The Nataf inversion theory is employed to generate source-load dual uncertainty power samples with correlation. Before each interactive training step, uncertainty power samples are randomly assigned to wind turbine nodes, and uncertainty load demand samples are added to load nodes to simulate real-world source-load uncertainty power system conditions. Subsequently, the SAC model obtains the current system state data from the simulation environment, selects an action based on an environmental state update policy, and delivers it to the simulation environment. After receiving the emergency frequency control action from the SAC model, the simulated power system environment executes the load adjustment action, advances to the next state, and sends the updated state data and immediate reward value to the SAC model. This training process continues until the end of a round, marked by maintaining stable system frequency. At this point, the system simulation environment is reinitialized, and the next round begins. Upon completing the training process, the SAC model can be applied to various fault test scenarios to validate its effectiveness and superiority.

The BPA software is utilized in this paper to generate a fault scenario for the IEEE10 machine with 39 nodes. The generator model is based on the sixth order model, while the load model consists of a constant impedance model and a mixed load model incorporating induction motors, with a 50% ratio between the two. The fault scenario involves a generator experiencing a partial power loss, resulting in a power difference within the power system. The total simulation time is 40 s, with each cycle of the waveform serving as a sampling point. To simulate various system fault states and obtain sufficient samples, one of the ten generators is randomly selected at the start of the simulation to experience a loss of active output ranging from 0.5 p. u. to one p. u.

This paper utilizes a modified version of the IEEE10 machine with 39 nodes to validate the proposed methodology in this section. The modification involves replacing nodes 32 and 36 with turbines having rated capacities of 684 MW and 576 MW, respectively. Additionally, nodes 3, 4, 7, 8, 16, 20, 24, and 39 are designated as controllable load nodes participating in frequency emergency control. The system’s topology is illustrated in Figure 4.

The power fluctuations at the load nodes follow a normal distribution with a mean and standard deviation equal to 5% of the rated value. Similarly, the load static model voltage and frequency indices also have a mean and standard deviation of 5% of the rated value.

The wind speeds of the wind nodes are modeled by a Weibull distribution with the shape parameter K set to 2.26, the scale parameter C set to 7.55, the cut-in wind speed at 3.5 m/s, the cut-out wind speed at 25 m/s, and the rated wind speed at 7.3 m/s.

To account for the correlation between the wind turbine nodes, 1,000 sets of wind turbine output samples are generated using the Nataf inverse transformations, with correlation coefficients of 0.8. Figure 5 illustrates the Weibull distribution of wind speed.

The deep reinforcement learning state space in this system comprises frequency deviation, frequency rate of change, active output, and load of each node, resulting in a 47-dimensional space. The action space consists of eight load shedding actions for controllable loads. Each action is represented as an 8-dimensional vector, where each element is a continuous value within the range of [-1, 0]. Furthermore, as the Soft Actor Critic (SAC) algorithm can handle continuous action spaces, the emergency frequency control directly determines the necessary load shedding amount and sets the action time for emergency frequency control as 2 s after fault detection. The delay characteristics of controllable loads are categorized into three levels. For loads of the same delay level, the actual control delay is calculated based on the maximum value to ensure that the actual frequency drop depth is less than or equal to the ideal frequency drop depth, thereby avoiding frequency instability. Consequently, after aggregation, it is assumed that the control delay for all level 1 controllable loads is 100 ms, for level 2 controllable loads is 200 ms, and for level 3 controllable loads is 300 ms. The controllable loads are then removed within each node in order of delay from low to high. Table 1 provides the proportions of controllable loads at each node and the distribution of loads across different control delay levels after aggregated modeling.

The policy network and value network of the SAC model both consist of two hidden layers with 64 neurons each. The activation function is set to ReLU, the learning rate is 0.005, the initial temperature coefficient is 0.1, the self-updating learning rate is 0.0001, and the updating algorithms utilize the alternating multiplier method. The experience replay unit has a capacity of 2,500, and 64 samples are drawn for each training iteration. The convergence criterion for each training round is that the absolute value of the steady-state frequency deviation is less than 0.1 Hz.

The SAC algorithm is employed to learn and train the aforementioned arithmetic model. Figure 6 depict the curves illustrating the changes in reward values during the training process.

Figure 6 demonstrate that, initially, the model struggles to find a control strategy that effectively stabilizes the system frequency, resulting in frequent movements per round and consequently low reward values. Additionally, the maximum number of action steps per round often reaches 50. However, as training progresses, the model gradually discovers more efficient control strategies with shorter action sequences, although the reward value remains suboptimal due to excessive load removal. It is only after 1,200 rounds of training that both the reward value and the number of training rounds stabilize, indicating the completion of the model training process.

To evaluate and compare the frequency recovery process of the proposed emergency frequency control scheme, it is essential to conduct tests using various fault scenarios. These scenarios are characterized by four attributes: the number of faulty nodes, the extent of power shortage in the faulty nodes, the system load factor, and the magnitude of source load fluctuations. For this purpose, four representative fault scenarios are selected, as illustrated in Figure 7.

During the model training process, the emergency frequency control policies for the four representative scenarios are derived through testing at intervals of 400 rounds until the completion of 2000 rounds, leading to the acquisition of the optimal control policy, as depicted in Figure 8.

Figure 8 clearly demonstrate significant fluctuations in the emergency frequency control strategies during rounds 0, 400, 800, and 1,200, indicating the model’s continuous search for an improved control strategy. In contrast, the control strategies for rounds 1,600 and 2000 exhibit reduced fluctuations, indicating that the model has undergone substantial training. Initially, the emergency frequency control strategy is more random, but through continuous training, the model takes into account factors such as the amount of controllable loads at each node and load removal sensitivity. Consequently, it selects an optimal node for load shedding, resulting in a final strategy with total load removal close to the power deficit.

Table 2 presents the controllable load shedding quantities for the optimal policy in the four representative test scenarios, along with the steady-state frequency values achieved post-policy implementation and the minimum value of dynamic frequency drop.

Table 2 reveals that in the four test scenarios, characterized by diverse fault locations, fault sizes, system loading rates, and source-load uncertainties, the trained model successfully maintains the system within 0.1 Hz of the steady-state frequency deviation. Additionally, the lowest point of the dynamic frequency drop remains above 49.5 Hz. These results substantiate the effectiveness of the emergency control strategy based on the SAC algorithm, particularly for systems affected by source-load uncertainties.

To further ascertain the superiority of the proposed method, a comparative analysis is conducted between the emergency frequency control strategy derived from the traditional adaptive UFLS algorithm and the strategy proposed in this paper. The dynamic frequency recovery process of the system is evaluated for both strategies across the four scenarios, as depicted in Figure 9.

Figure 9 demonstrates that the emergency frequency control strategies optimized by the proposed scheme in this paper effectively maintain the steady-state frequency deviation of the system within 0.1 Hz, with the lowest frequency point exceeding 49.5 Hz across the four different scenarios. In contrast, the adaptive UFLS scheme in Scenarios 1, 2, and three suffers from the issue of insufficient load shedding, resulting in a greater depth of frequency drop and steady-state frequency deviation. Additionally, the conventional scheme in Scenario four exhibits excessive load shedding, leading to a steady-state frequency close to 50.4 Hz. Consequently, the method presented in this chapter proves its superiority in reducing the depth of frequency drop and steady-state frequency deviation, highlighting the effectiveness of the deep reinforcement learning algorithm.

To compare the disparities between source-load uncertainty and deterministic power systems, both the conventional method and the SAC algorithm proposed in this chapter are employed in both systems for 100 tests. The emergency frequency control outcomes are then compared, and the results are illustrated in Figure 10.

Figures 10A, B reveal that the median frequency nadir achieved by the SAC algorithm in the source-load deterministic system and the uncertain system is approximately 49.65 Hz and 49.6 Hz, respectively, whereas the median values obtained by the traditional method are around 49.55 Hz and 49.45 Hz, respectively. Notably, the frequency nadir resulting from the traditional method is significantly lower than that achieved by the deep reinforcement learning method, making it nearly impossible to maintain system frequency stability in numerous scenarios. By contrast, the SAC algorithm effectively improves the steady-state frequency deviation and frequency nadir in both deterministic and uncertain systems, demonstrating its superiority over the traditional method for addressing the emergency frequency control problem in source-load uncertain systems.

To validate the suitability of the SAC algorithm over other reinforcement learning algorithms for addressing the emergency frequency control problem in the source-load double uncertainty system, the model developed based on the SAC algorithm in this paper is compared with models employing the A2C algorithm and the TD3 algorithm. Figure 11 presents a comparison of the reward value’s increasing trend throughout the training process. The solid line represents the smoothed reward value, while the shaded area denotes the variance fluctuation of the reward value.

Figure 11 illustrates that after approximately 500 rounds, the smoothed reward value of the model based on the SAC algorithm surpasses that of the other algorithm models, exhibiting a gradual increase until it stabilizes at the desired value. Furthermore, in terms of variance, the reward value’s variance for the SAC algorithm is higher during the initial 300 training rounds and subsequently becomes smaller than that of the other two algorithms. This observation indicates the robustness of the SAC algorithm, its ability to swiftly enhance the reward value through learning, and its reduced oscillation.

The SAC algorithm effectively decreases the minimum system frequency drop compared to other DRL algorithms, while also reducing the steady-state frequency deviation. To visually demonstrate the test’s improvement more intuitively, Figure 12A and (B) present the distribution of frequency drop nadir and steady-state frequency deviations resulting from the tests conducted with various algorithms under random scenarios.

As can be seen from Figure 12, the test results of the emergency frequency control strategy using the SAC algorithm show that the probability of the system’s steady-state frequency stabilizing at 49.8Hz–50 Hz is more than 50%, which is much higher than that of the test results using the A2C and TD3 algorithms, and the probability of the frequency dip nadir of the SAC algorithm being higher than 49.4 Hz is much higher than that of the other two algorithms. Therefore, the model based on SAC algorithm in this chapter can effectively improve the dynamic frequency nadir and steady-state frequency of the system after emergency frequency control compared to other DRL algorithms.