Before the advent of ResNet, popular neural network models such as AlexNet (Krizhevsky et al., 2012) and VGG (Simonyan et al., 2014) learned the target features more effectively by simply stacking the number of neural network layers (Figure 1A). However, when the number of neural network layers reaches a certain depth, issues such as gradient disappearance, gradient explosion, and overfitting will occur, resulting in network degradation, lower training accuracy, and lower test accuracy. By adding a residual module, ResNet addresses network degradation by reducing gradient disappearance, gradient explosion, and overfitting caused by an excessively large number of neural network layers.

The basic structure of the residual module is shown in Figure 1B. Assuming that the neural network’s input is x, its predicted output is H(x), and its actual output is G(x). Unlike the conventional convolution module A structure, in the residual network, the input x can be directly passed to the output as the initial result by skip connections, and the output result becomes H(x) = G(x) + x. This is equivalent to the residual module changing the learning target and no longer learning a complete output, but instead learning the difference between the target value H(x) and x, (i.e., the residual G(x) = H(x) - x).

The biggest difference between ResNet and other CNNs is the addition of identity mapping. If the neural network contains more network layers than the recommended number, the residual network will train the redundant layers to G(x)=0, which means that their inputs and outputs are the same, thereby changing the network into an identity map. Gradient explosion or disappearance during network training can be avoided by using the identity mapping function, which allows the network to quickly transfer the gradient value from deeper to shallower layers. Therefore, adding more layers to the network will not result in network degradation, thereby enhancing the stability and effectiveness of the network training process.

GANs differ from regular neural networks, as they comprise generator and discriminator networks. GANs originated from the two-person zero-sum game in game theory. Through cooperative games, both the generator and the discriminator improve at their respective functions. The value of the loss function decreases over time as the model parameters are continuously optimized and iterated during training, eventually reaching Nash equilibrium, which is a state of equilibrium in the game process.

CycleGAN was enhanced based on GANs and contains two generators and two discriminators. Cycle consistency was the central concept. The data Generated_Y can then obtain the same data Cyclic_X as the data Input_X through the generator Y2X (Figure 2). This ensures that Input_X is consistent with Cyclic_X, which is equivalent to making the data cycle back to the starting point and maintaining its consistency. The original data is Domain X, the target data is Domain Y, the generator that transfers data from Domain X to data in Domain Y is G, and the generator that transfers data from data in domain Y to data in domain X is F. The discriminator that determines whether or not the data belongs to Domain X and is real is D _X , and the discriminator that determines whether or not the data belongs to Domain Y and is real is D _Y .

First the loss that corresponds to cycle consistency is defined, also known as the cycle-consistency loss. The representation of CycleGAN is as follows to guarantee cycle consistency:

When converting from data in Domain X to data in Domain Y: x → G x → F G x ≈ x

When converting from data in Domain Y to data in Domain X: y → G y → F G y ≈ y

This can be stated as a mathematical formula: L c y c G , F = E x ∼ P d a t a x F G x − x 1 + E y ∼ P d a t a y G F x − y 1 (1)

GAN loss is the loss function corresponding to a regular GAN, in addition to the formula corresponding to cycle-consistency loss. The loss function of the GAN composed of G and D _Y is: L G A N G , D Y , X , Y = E y ∼ P d a t a y log D Y y + E x ∼ P d a t a x log 1 − D Y G x (2)

The loss function of a GAN composed of F and D _X is: L G A N F , D X , Y , X = E x ∼ P d a t a x log D X x + E y ∼ P d a t a y log 1 − D X G y (3)

Instead of the conventional GAN loss, we propose to use the least square GAN squared difference loss. Traditional GAN generates less accurate results if cross entropy is used as the loss function, as the generator will stop optimizing the data that the discriminator believes to be true, even if they differ from the real data. This yields seismic data with a low quality, and its denoising effect is not readily apparent. The improved CycleGAN method proposed herein uses the least square GAN to replace the objective function of a traditional GAN, using the squared difference as the loss instead of the Log-likelihood to improve the feature extraction effect and obtain better in - phase axis texture preserving effect. Its formula is as follows: L L S G A N G , D Y , X , Y = E y ∼ P d a t a y D Y y − 1 2 + E x ∼ P d a t a x D Y G x 2 (4) L L S G A N F , D X , Y , X = E x ∼ P d a t a x D X x − 1 2 + E y ∼ P d a t a y D X G y 2 (5) When these three components are added together, the total loss objective function of CycleGAN is obtained: L G , F , D X , D Y = L L S G A N G , D Y , X , Y + L L S G A N F , D X , Y , X + λ L c y c G , F (6)

The experimental workflow used in this study is shown in Figure 5. A sample set of seismic data was created for network training after a series of pre-processing steps, which included normalizing and standardizing the seismic data. The improved CycleGAN network was trained using a sample set of original seismic data, as well as a sample set of seismic data with added random noise. The final results of the seismic data denoising process were evaluated and analyzed using signal-to-noise ratio (SNR), peak SNR (PSNR), and structural similarity (SSIM) quantitative parameter indicators. It was necessary to change the network parameters, retrain, and test until an optimized network model was attained if the training and testing results were unsatisfactory.

In this study, model and real seismic data were used to verify the denoising effect of the improved CycleGAN network. To evaluate the denoising effect, quantitative indicators were added, in addition to qualitative analysis. As evaluation indicators for data denoising, SNR (Zhang et al., 2009), PSNR (Netravali, 1995), and SSIM (Wang et al., 2004) were used. S N R = E s E n = ∑ i = 0 N − 1 s 2 t ∑ i = 0 N − 1 g t − s t 2 (7) The SNR is the ratio of the effective signal energy to the noise energy, which is the square of the amplitude ratio. E _s is the energy of the effective signal, E _n is the energy of the noise, s(t) is the energy of the effective signal, and g(t) is the energy of the denoised signal. Generally, the denoising effect improves and the residual noise energy of the denoised signal decreases as the SNR increases.

The root mean square error is expressed by the following: M S E = 1 m n ∑ i = 0 m − 1 ∑ j = 0 n − 1 I i , j − K i , j 2 (8) where I is the seismic data without random noise, K is the seismic data with random noise, and m and n are the data sizes among them. After obtaining the root mean square error, the PSNR is expressed as: P S N R = 201 g 2 n − 1 2 M S E (9)

Generally, higher PSNR values indicate a more obvious denoising effect. S S I M x , y = 2 μ x μ y + c 1 2 σ x y + c 2 μ x 2 + μ y 2 + c 1 σ x 2 + σ y 2 + c 2 (10) where x and y are seismic data with random noise and the denoised results, respectively; μ _x is the mean of x; μ _y is the mean of y; σ2 x and σ2 y are the variances of x and y, respectively; and σ _xy is the covariance of x and y. C ₁ and C ₂ are constants that maintain the stability of the structure, i.e., the denominator is guaranteed to be non-zero. Generally, larger SSIM values indicate higher data similarity.

The model data in this study simulated the generation of seismic datasets using the methods of Wu et al. (2020), which produced synthetic 3D seismic data that is similar to real seismic data (Figure 6). To create a two-dimensional data sample, 128 data traces were selected in the crossline direction and 128 sampling points were selected in the time direction. Then, 2,600 data traces were selected in the inline direction as the sample set, to which random noise was added to create 2,600 groups of sample sets. As shown in Figure 7, there was no 1:1 correspondence between the original model sample data and the model sample data that contained random noise (SNR is 0.2673). After a series of data pre-processing operations that included data normalization and standardization, the sample set was input into the improved CycleGAN. Training was performed in the network, and the final training effect was tested. The ratio of the training dataset to the test dataset was set to 8:2. The final test results were compared with those of FX deconvolution denoising method and original CycleGAN denoising methods. For comparison, the SNR, PSNR, and SSIM were used as evaluation indicators for the data denoising effect. During training, we set the initial learning rate to 0.0002, used the Adam optimization algorithm, and set the number of epochs to 2000. A dedicated computer (Intel Xeon Gold5188) with 3584 CUDA cores, 16 GB of video memory, and an NVIDIA Tesla P100 graphics card was used to train the model. Figure 8 shows the improved CycleGAN loss function curve. It can be seen that the network converges after about 1,500 epochs. The denoising effect is shown in Figure 9.

Although some random noise was removed from the denoised data produced by FX deconvolution denoising method, the overall denoising effect was not clear and some random noise persisted, which masked the useful information contained in the seismic data (Figure 9). Subsequent interpretation of the seismic data presented challenges. The original CycleGAN method still had an ambiguity issue in the description of the effective information details after the random noise was removed; however, the denoising effect was improved significantly compared to that of FX deconvolution denoising method. The improved CycleGAN network clarified the event information of the seismic data output by the generator and enhanced the texture features by combining the U-Net and ResNet structures. To improve the final data output quality, the PatchGAN discriminator was used to separate each detail of the data output by the generator. The seismic data event information obtained by the improved CycleGAN denoising method was clearer and the texture features were more obvious compared to FX deconvolution denoising method and original CycleGAN denoising methods (red box in Figure 9). The improved CycleGAN denoising method also had better denoising and effective information retention effects on the seismic data (Table 1).

We use publicly accessible three-dimensional seismic data from the Netherlands F3, which were collected in the offshore North Sea region and comprise a large dataset. Figure 10 shows the crossline (100–611) and inline (300–683) direction ranges, as well as the time range (1,337–1,848 ms) and sampling interval (4 ms). To create a two-dimensional sample of the data, 384 lines of data were selected in the inline direction and 512 sampling points were selected in the time direction. Then, 120 data lines were selected in the crossline direction as the sample set. The sample data were expanded to a total of 1,000 sample sets using operations including up and down flip, left and right flip, and interpolation. A specific amount of random noise was then added to the sample data to create 1,000 sample sets with random noise. The improved CycleGAN network was trained using the sample datasets mentioned above. Another section with substantial random noise added to the seismic data was also tested.

Figure 11 demonstrates that FX deconvolution denoising method did reduce some of the random noise to a certain extent, issues like masked event information and fuzzy texture details were still present. The denoising effect of the original CycleGAN method was significantly enhanced compared to that of FX deconvolution denoising method on the real seismic data. Although a substantial amount of random noise was eliminated, issues with the detailed characterization of the seismic data remained, as well as discontinuous issues, unclear expressions of the in-phase axis information, and lost texture detail features. In addition to having a better effect on removing random noise from the real seismic data, the improved CycleGAN method proposed herein also better expressed the intricate features of the real seismic data. The improved CycleGAN method depicted the event axis information more clearly and continuously, the event edge information restoration effect was improved, and the texture detail features were clearer and richer (red box in Figure 11). In addition, the improved CycleGAN method retained more useful information for the real seismic data and had a more noticeable denoising effect than FX deconvolution denoising method and original CycleGAN denoising method (Table 2).