An AGC Dynamic Optimization Method Based on Proximal Policy Optimization - Liu, Zhao; Li, Jiateng; Zhang, Pei; Ding, Zhenhuan; Zhao, Yanshun (Frontiers in Energy Research, 2022)
The increasing penetration of renewable energy introduces more uncertainties and creates more fluctuations in power systems than ever before, which brings great challenges for Automatic Generation Control (AGC). It is necessary for grid operators to develop an advanced AGC strategy to handle fluctuations and uncertainties. AGC dynamic optimization is a sequential decision problem that can be formulated as a discrete-time Markov decision process. Therefore, this paper proposes a novel framework based on Proximal Policy Optimization (PPO) reinforcement learning algorithm to optimize power regulation among each AGC generator in advance. Then, the detailed modeling process of reward functions, state and action spaces designing is presented. The application of the proposed PPO-based AGC dynamic optimization framework is simulated on a modified IEEE 39-bus system, and compared with the classical proportional−integral (PI) control strategy and other reinforcement learning algorithms. The results of the case study show the framework proposed in this paper can make the frequency characteristic better satisfy the Control Performance Standard (CPS) under the scenario of large fluctuations in power systems.