The dynamic model of the PEMFC has been refined from the electrochemical model. Ideally, the voltage released at full reaction is 1.229 V. The actual potential decreases due to irreversible losses, which in practice are also known as polarization overvoltage. In the power generation process of a PEMFC, polarization overvoltage is mainly manifested as activation overvoltage, ohmic overvoltage and concentration overvoltage. Therefore, in the actual power generation process, the individual voltage is inevitably less than the ideal standard electric potential due to the polarization overvoltage. In addition to the factors such as temperature, pressure and current density, chemical and material factors such as electrode material and electrolyte can also influence the polarization or overvoltage of the electrodes. V cell  = E − η act  − η ohm  − η con  (1) For a fuel cell stack consisting of N single cells connected in series, the output voltage V can be expressed as V = N V cell  (2)

The ohmic polarization overvoltage is mainly caused by the equivalent membrane impedance of the proton exchange membrane to the transfer of protons and the impedance of the electrodes and current collectors to the transfer of electrons. Based on the Amphlett model, the PEMFC ohmic overvoltage mainly includes the voltage drop caused by the impedance of the two parts of the PEMFC. These two parts of impedance, one part is the equivalent membrane impedance of the proton membrane, The other part is the resistance that prevents protons from passing through the proton membrane which is usually a constant. According to the resistivity theorem, the equivalent membrane impedance R _m can be obtained by the following formula: R M = ρ M B A (3) In the formula, ρ ( M ) is the resistivity of the proton membrane to the electron flow ( Ω ⋅ ( cm ) ) , B is the thickness of the proton exchange membrane. ρ M can be obtained by the following formula. ρ M = 181.6 [ 1 + 0.03 ( I A ) + 0.062 ( T 303 ) ( f r a c I A ) 25 [ λ − 0.634 − 3 ( I A ) ] exp [ 4.18 ( T − 303 T ) ] (4) Empirically, the internal resistance of the battery is R int = 0.01605 − 3.5 × 10 − 5 T + 8 × 10 − 5 i (5) Ohmic polarization overvoltage can be expressed as: η ohm  = I R int  = I ( R m + R c ) (6)

Activation overvoltage is the deviation of an electrode’s potential from its equilibrium potential due to a delay in its electrochemical reaction. The activation polarization overvoltage of the cathode can be obtained V ux  t , c   = R T α c F n ( ln [ n F K c 0 ⁡ exp ( − Δ G R T ) ×   ( C i 2 ) ( 1 − α i ) ( C H 2 ) ( 1 − α c ) ( C H 2 o ) α ι ] − ln ⁡ I )   (7) Anode activation polarization overvoltage: V a t , a = − Δ G 2 F + R T 2 F ln ( 4 F A K a 0 C H 2 ) − R T 2 F ln ⁡ I (8) The total activation polarization overvoltage is the sum of the anode activation overvoltage and the cathode activation overvoltage, expressed as: η act = ξ 1 + ξ 2 T + ξ 3 T ⁡ ln ⁡ c ( O 2 ) + ξ 4 T ⁡ ln ⁡ I (9) c ( O 2 ) is the concentration of dissolved oxygen at the cathode catalyst interface which is calculated as follows: c ( O 2 ) = P O 2 / 5.08 × 10 6 ⁡ exp ( − 498 / T ) (10)

According to the empirical formula for the PEMFC, the thermodynamic electric potential can be obtained as follows: E = Δ G 2 F + Δ S 2 F ( T − T ref ) + R T 2 F ( ln ⁡ p H 2 + 1 2 ln ⁡ p 0 2 ) (11) PEMFC can directly convert chemical energy into electrical energy. The chemical energy release of a fuel cell can be calculated by the change of the Gibbs self-burning energy ( Δ g ( f ) ) . The Gibbs self-burning energy is usually used to calculate the externally available energy, for basic chemical reaction formula of hydrogen/oxygen reaction PEMFC is: H 2 + 1 2 O 2 − H 2 O (12) The corresponding change in Gibbs’ self-reliance is: Δ g f = g f ,   p r o d u c t s   − g f ,   r e a c t a n   i s   = ( g f ) H 2 O − ( g f ) H 2 − ( g f ) O 2 (13) The changed Gibbs self-burning energy is a function of temperature and pressure: Δ g f = Δ g f 0 − R ¯ T f c ⁡ ln [ p H 2 p O 2 1 2 p H 2 O ] (14) We can deduce the voltage of the fuel cell: E = − Δ g f 2 F = − Δ g f 0 2 F + R T f c 2 F ln [ p H 2 p O 2 1 2 p H 2 O ] (15) When specific values are brought in, the equation can be transformed into E = 1.229 − 0.85 × 10 − 3 ( T − 298.15 ) + 4.3085 × 10 − 5 T ( ln ⁡ p H 2 + ln ⁡ p 0 2 / 2 ) (16)

Concentration overvoltage is a phenomenon caused by the deviation of the electrode potential from the equilibrium potential due to the difference between the concentration of ions in the solution at the electrode interface layer and the concentration of the body solution in the electrolytic bath, which can be expressed as: η con = − β ⁡ ln ( J / J max ) (17)

The phenomenon of a “double layer of charge” in a proton exchange membrane fuel cell is particularly important for the dynamics of the PEMFC. On the surface of the electrode electrons are collected and on the surface of the electrolyte hydrogen ions are collected. Between them there is a potential difference in which charge and energy are stored, which acts as an equivalent capacitance. This “smoothest out” the voltage loss across the equivalent resistance and results in a very realistic dynamic model of the PEMFC. Therefore, when modelling the dynamics of the PEMFC, a capacitance is added to the electrochemical model. This “equivalent capacitance” is able to better represent this effect by smoothing the output voltage response of the fuel cell as the current changes, with a transition time.

In Figure 1, the polarization voltage across _Rd is V d , given by the differential equation for the voltage change of a single cell as d V d / d t = I / C − V d / R d C (18) V d = i R d = η a c t + η c o n (19) Thus, the voltage of the stack can be expressed as: V s t = n V cell  = n [ E − I ( R a c t + R con  + R ohm  ) ] = n ( E − I R equ  ) (20) The output power and efficiency of the stack can therefore be expressed as: P s t = I ⋅ V s t η = V cell  ⋅ I / E ⋅ I = V cell  / E (21)

The DDPG method fuses deep neural networks with Deterministic Policy Gradients (DPG) algorithms and uses actor -critic a framework as the basic architecture for the algorithm. The actor network is used to update the policy and the critic network is used to approximate the state action value function. The use a non-linear neural network as an approximator. Inspired by the algorithm, DQN solve this problem by setting up a target actor network and a target critic network, as well as an experience replay mechanism. Instead of DQN directly copying the current network to the target network, DDPG updates the target network in a “soft” way, ensuring that each parameter update is small, thus achieving a stable training effect. θ Q ′ ← τ θ Q + ( 1 − τ ) θ Q ′ θ μ ′ ← τ θ μ + ( 1 − τ ) θ μ ′ (22) In DDPG, the objective function is defined as a sum with discounted rewards J ( θ μ ) = E θ μ [ r 1 + γ r 2 + γ r 3 + ⋯ ] (23) L ( θ Q ) = 1 N ∑ i ( y i − Q ( s i , a i | θ Q ) ) 2 (24) Of which y i = r i + γ Q ′ ( s i + 1 , μ ( s i + 1 | θ μ ′ ) | θ α ) (25)

The actor network parameters are updated by means of a chain derivative rule for the objective function: ∇ θ a J ( θ μ ) ≈ 1 N ∑ i ∇ a Q ( s , a | θ ϱ ) | s = s , a = μ ( x )   ∇ θ a μ ( s | θ μ ) | s j (26) To address the problem of under-exploration caused by actors mapping states to deterministic actions in the DPG approach, the DDPG algorithm generates temporal correlated noise through the Ornstein-Uhlenbeck (OU) process to improve the exploration capability of the algorithm under deterministic strategies.

DDPG uses an empirical replay mechanism based on random sampling but suffers from Q-value overestimation.

The IDDPG algorithm uses two critics. The target value formula is as follows: y t = r ( s t , a t ) + γ min i = 1,2 Q θ t ( s t + 1 , π ϕ ′ ( s t + 1 ) + ε ) (27)