In order to constrain the dependence of spatial distribution between the generated clean and noise distribution image, we introduce a reconstruction loss. Based on the assumption that the original image can be constructed by the output clean image and noise image, the reconstruction loss Lrecon is defined as: (2)Lrecon=12||Iin-(Nout+Cout)||F, where ||·||_F represents the Frobenius norm, and Lrecon enables the output noise distribution and clean image, respectively to share the consistency in spatial distribution with the input image.

Wu et al. (18) confirmed the existence of low-rank prior in stripe noise. According to the low-rank characteristic of stripe noise, the number of singular values of stripe-noise images should be approximated to zero after singular value decomposition. In order to keep the consistence between the generated stripe-noise map and the reference stripe-noise image of a synthesized image, we introduce a new stripe loss Lstripe: (3)Lstripe=||Stripe(Nref)-Stripe(Nout)||F, where N_ref represents the reference stripe-noise image.Stripe(·) is the stripe degradation function aimed at reconstructing the primary stripe component via a soft-thresholding operation on singular values, which is denoted as: Stripe(N) = U · shrink(S) · V^T, where N = U · S · V^T is singular value decomposition on N. shrink(·) is the soft-thresholding operation aimed at selecting non-zero singular values on S: (4)shrink(S)=diag{max(Sii-λ,0)}, where S_ii represents the diagonal element of singular value matrix S, and λ is a small positive constant for selecting singular values.

Vessel structures are the most significant biomarkers in OCTA or AS-OCTA images for clinical diagnosis, which are prone to be corrupted during the de-striping process. To this end, ATV loss is introduced to maintain the completeness of vessel structures. ATV is able to measure the edge sharpness of an image (42) and has been widely applied in image restoration with edge preservation. The proposed ATV loss constrains the edge of two images and is defined as follows: (5)LATV=||ATV(Cref)-ATV(Cout)||F, where C_ref represents the reference clean image; ATV(·) is the function to extract gradient information defined as: (6)ATV(C)=||▽uxC||1+||▽uyC||1 where ||·||₁ represents the L₁ norm; ▽_ux and ▽_uy represent the horizontal and vertical difference operations on the image C respectively.

Note that the proposed SR-Net was trained on our synthetic OCTA datasets, which contain the reference stripe-noise images N_ref, the reference clean images C_ref and corruption images synthesized from the both.

CycleGAN will prevent two generators from contradicting each other by converting unpaired learning into paired learning via constructing a self-supervisory signal. However, image-level cycle-consistency of CycleGAN is not adequate to focus on both low- and high-level features, which will lead to unsatisfactory artifacts in the enhanced image. To overcome this shortcoming, extra feature-level cycle-consistency called cyclic perceptual loss is introduced in the objective function. Different from image-level cycle-consistency, cyclic perceptual loss calculates cycle-consistency loss based on low- and high-level features extracted from the VGG-19 (60). Cyclic perceptual loss is defined as: (7)Lp(GXY,GYX)=∑l=2,5||ϕl(x)-ϕl(GYX(GXY(x)))||F2                          +||ϕl(y)-ϕl(GXY(GYX(y)))||F2 where x ∈ X, y ∈ Y, and ϕ^l(·) represents the output of the l-th max pooling layer of the VGG-19 feature extractor pretrained on the ImageNet. In Equation (7), features extracted from the 2nd and 5th max pooling layer of the pretrained VGG-19 are treated as low- and high-level ones, respectively to calculate cyclic perceptual loss.

In order to preserve vessel structures of the enhanced images, we utilized the structure loss (20) to maintain the invariance of vessel structures. Inspired by the structure comparison function in structural similarity (SSIM) metrics, the structure loss measures the dissimilarity of structure between the original image and its enhancement in local windows and is defined as follows: (8)Ls(G,X)=Ex∈X[1-1M∑i=1Mσxi,G(x)i+cσxiσG(x)i+c], where M represents the number of local windows of the input image, x_i and G(x)_i represent the i-th local window of an image and its generated one respectively; σ_xi and σ_G(x)i represent the standard deviations of x_i and G(x)_i respectively; σ_xi,G(x)i represents the covariance of x_i and G(x)_i; c is a small positive constant.

Finally, the loss function of PS-GAN can be expressed as: (9)   LPS(GXY,GYX,DX,DY)=Lbl(GXY,GYX,DX,DY)+ξLp(GXY,GYX)+ρ1Ls(GXY,X)+ρ2Ls(GYX,Y) where Lbl represents the baseline objective function of PS-GAN; ξ, ρ₁ and ρ₂ are the weighted parameters of each term except for Lbl.

We used the training set (n = 180) from the PUTH to train our SR-Net. The Adam optimizer with recommended parameters was used to optimize the model and batch size was set as 16. The initial learning rate was 2 × 10⁻⁴ and gradually decayed to zero after 300 epochs. The hyper-parameters used in the network were λ = 0.002, α = 0.5, β = 2 and γ = 1.

Due to the lack of high-quality AS-OCTA images for training, and the high similarity of contrast and structural distribution between AS-OCTA image and corneal confocal microscopy, in this work, we combined the training set (n = 180) of the PUTH and a corneal confocal microscopy dataset (CCM) to form a new dataset to train our model. The public-accessible dataset CORN-2 (20) was used, and it contains unpaired 340 low-quality and 288 high-quality CCM images. In the experiment, domain X consists of low-contrast CCM, and all the images in PUTH and the high-quality images in CORN-2 were included in domain Y. All the images in both domains were resized to 400 × 400. The proposed PS-GAN was optimized using Adam and batch size was set to 1. The initial learning rate was 2 × 10⁻⁴ for the first 100 epochs and gradually decayed to zero after the next 100 epochs. The hyper-parameters used in the network were ξ = 0.5, ρ₁ = ρ₂ = 0.5.

As we mentioned in the dataset section, synthetic stripes were added to the images in the PUTH and ROSE datasets. In order to demonstrate the superiority of the de-striping stage, we compared our SR-Net with the state-of-the-art de-striping methods including CUD, CSD and CSD+ (18), as well as the baseline model (U-Net). Figures 7, 8, top present the stripe noise removal results produced by the different methods. As illustrated in Figure 7, top, SR-Net removes stripe noise more faithfully without losing vessel information comparing with other methods. CSD+ attempts to remove the effect of stripe artifacts from the given images, but it still contains noticeable stripe artifacts. Overall, the proposed SR-Net generate the best performance in eliminating stripe noise, i.e., with more visually informative results.

It is difficult to demonstrate conclusively the superiority of the enhancement method purely by the above visual inspection, in this section, a quantitative evaluation of de-striping is provided. The PSNR and SSIM values of different methods on both the PUTH and ROSE datasets are shown in the Table 1. As we can see, the higher stripe noise level (i = 1, 2, 3, 4, the larger i, the stronger corruption) usually lead to the lower PSNR and SSIM. It is evident that our SR-Net achieves the highest PSNR and SSIM under various levels of noise corruption. For those OCTA images with low-level stripe noise, all de-striping methods could achieve relatively high performance, and the proposed SR-Net achieves a slightly higher PSNR and SSIM scores than other approaches. For example, SR-Net achieves the improvements of only 8.98, 8.86, 9.16, 1.08, 1.08, and 0.21% when, respectively, compared with CUD, CSD, and CSD+ in terms of PSNR and SSIM on ROSE dataset at noise level i = 2. By contrast, the performance of CUD, CSD and CSD+ declines and the proposed SR-Net yields better performance with relatively more significant margins for those OCTA images with high-level stripe noise. For example, the proposed SR-Net achieves an increase of about 31.72, 31.71, 36.63, 7.01, 7.01, and 9.11% in terms of PSNR and SSIM when, respectively, compared with CUD, CSD, and CSD+ at noise level i = 4. Similarly, our method also yields better performance with large margins on the PUTH dataset when the images were corrupted by high-level noise. It indicates that the proposed SR-Net is robust to different stripe noise levels.

We also perform vessel segmentation over de-striped images to confirm the relative benefits of the proposed method in comparison to the others. For vessel segmentation in OCTA images, we employed a recent proposed vessel segmentation network which is designed for OCTA images in the ROSE dataset: OCTA-Net (4). We utilized OCTA-Net to perform vessel segmentation on images with synthetic stripe noise and images after applying different de-striping methods.

The Figure 8, bottom shows the vessel segmentation results by different method on a noise corrupted image. The benefit of the proposed SR-Net method for segmentation may be observed from the representative region (red patches). It can be seen that more accurate and completed visible vessels have been identified by our SR-Net compared with original images with stripe noise and other destriping methods despite difficulties in small vessel detection. By contrast, a large portion of stripe noise has been identified as vessels by OCTA-Net in synthetic images and the de-striped images after applying other de-striping methods. This finding is also evidenced by segmentation results reported in Table 2. It is indeed that our SR-Net improves the segmentation performances when compared to the results of synthetic images: by an increase of about 7.79, 3.40, 7.80, and 8.42% in Dice and 2.71, 1.33, 2.96, and 3.59% in G-mean, respectively with various levels of stripe noise added in the original images. Moreover, when compared with CUD method, SR-Net achieves improvement of about 1.06, 0.79, 1.06, and 1.33% in Dice, 0.72, 0.72, 0.72, and 1.21% in G-mean, respectively, with various levels of stripe noise added. When compared with CSD method, SR-Net achieves improvement of about 1.06, 0.93, 1.06, and 1.33% in Dice, 0.60, 0.84, 0.72, and 1.09% in G-mean with the corruption of different noise levels. Similar to the results of PSNR and SSIM, vessel segmentation performance gain with larger margin has also been achieved by the proposed SR-Net when the images were corrupted by high-level noise.

For vessel segmentation in AS-OCTA dataset, we employed the Curvelet denoising based Optimally Oriented flux enhancement method (COOF) (61), which was proposed for segmentation of retinal microvasculature in OCTA images. It is worth noting that, on one hand, no pre-trained deep learning models may be utilized to performance the segmentation in AS-OCTA dataset, due to the lack of high-quality AS-OCTA images. On the other hand, it is our purpose to validate the proposed de-striping method would benefit both deep learning-based and conventional segmentation methods.

As shown in Figure 9, bottom and the representative patches (red box), it may be seen clearly that COOF is able to identify vessel structures more accurately in the images with less stripe noise falsely detected as vessels after our SR-Net is applied. We can observe that the competing methods have retained some stripe noise in Figure 9, top and the representative patches (yellow box), and the appearance of stripe and vessels are of great similarity, thus the COOF method falsely detects stripe noise as vessels. By contrast, most stripe noise similar to vessels has been well removed by the proposed SR-Net. Furthermore, Table 4 demonstrates the superiority of the proposed SR-Net in improving segmentation performances. It shows that our method yields the highest scores in terms of Dice, Sen, and G-mean. SR-Net achieves G-mean of 0.714, with an improvement of about 5.78%, 5.62%, and 3.03% over CUD, CSD, and CSD+.

In summary, the proposed SR-Net is helpful in improving the accuracy of vessel segmentation, since more stripe noise has been removed, and it would enhance the visibility of the vascular structure for subsequent processing.

In order to demonstrate the superiority of the re-enhancing stage, we compared our PS-GAN with several state-of-the-art image enhancement methods, including CycleGAN (59), EnlightenGAN (56), CycleDehaze (62), and StillGAN (20). All these competing methods are based on GAN, which aim to translate an image from low-quality to high-quality domain in this work.

The Figure 10, top presents the visual enhancement results via different methods. The results obtained by EnlightenGAN and CycleDehaze have image distortion and blurring affect because these methods are hard to guarantee the preservation of details in the images. CycleGAN and StillGAN produced relative better results than EnlightenGAN and CycleDehaze, however, they showed some side effects such as intensity inhomogeneity and vessel discontinuity. In contrast, the proposed method yielded more visually informative results, i.e., relatively visual pleasing contrast and clear visibility of vessel structures.

We also calculated PSNR and SSIM for quantitative evaluation of re-enhancing stage over the PUTH and ROSE datasets. The quantitative results of different enhancement approaches are shown in Table 3. The proposed PS-GAN obtained the best performances in terms of both metrics - it shows large margin when compared with EnlightenGAN and CycleDehaze as they have changed the content of the given images.

For example, PS-GAN achieves higher PSNR and SSIM scores in ROSE dataset when compared with EnlightenGAN, CycleDehaze and StillGAN if we added different levels of stripe noise, with 158.83, 54.16, 6.52, 100, 30.61, and 0.84% higher when compared with competing methods in terms of PSNR and SSIM when noise level i = 2. Similarly, our method also yields better performance with large margin in the PUTH dataset when the images were corrupted by different noise levels. These findings also demonstrate the robustness of PS-GAN to different stripe noise levels.

In order to confirm the impact of the re-enhancing stage on vessel segmentation, we further performed vessel segmentation and compared segmentation results from the enhanced images obtained by different methods on the AS-OCTA dataset.

For the vessel segmentation in the AS-OCTA dataset, we also employed COOF (61) as the vessel segmentation method. The Figure 10, bottom presents the segmentation results via COOF, it may be seen that COOF is able to identify vessel structures more accurately on the images after our PS-GAN is applied. Because of the hard presentation of details in the images, the segmentation results of EnlightenGAN and CycleDehaze are deviated from the ground truth. CycleGAN and StillGAN produced relative better segmentation results than EnlightenGAN and CycleDehaze, however, they showed some side effects such as intensity inhomogeneity and vessel discontinuity of vessel segmentation results. In contrast, the proposed PS-GAN yielded more visually informative results, COOF detects more vessel structures. Furthermore, Table 5 demonstrate the obvious advantage of the proposed PS-GAN in improving Dice, Sen and G-mean of vessel segmentation compared with other competing approaches. Moreover, our PS-GAN achieves G-mean of 0.812, with an improvement of about 12.62%, 13.57% and 8.12% over EnlightenGAN, CycleDehaze and StillGAN, respectively.

In order to verify the effectiveness of our proposed method for the stripe noise removal, we regarded the U-Net which shares the same structure with SR-Net and adopts mean square error (MSE) loss function as the baseline model and confirmed the stripe removal effectiveness of the proposed loss function L_destripe.

Visually For synthetic OCTA images in PUTH and ROSE datasets, comparative results of the baseline and our SR-Net are illustrated in Figure 7 and the top row of Figure 8, indicating that the baseline model still contains some noticeable stripe artifacts whilst our SR-Net can remove stripe artifacts satisfactorily. As illustrated in Figure 9, top in AS-OCTA, our SR-Net removes stripe artifacts more cleanly than the baseline, in addition, our method will not lose vessel information.

Vessel segmentation For synthesis OCTA with annotations in the ROSE dataset, the vessel segmentation results of baseline and SR-Net are shown in Figure 8, bottom, our method preserves vascular integrity better. This finding is also evidenced by the segmentation results reported in Table 2. When compared with the baseline, the proposed SR-Net achieves improvement of about 0.53, 0.13, 0.53, and 0.53% in Dice, 0.24, 0.12, 0.00, and 0.60% in G-mean with the corruption of different noise levels. For the AS-OCTA dataset, as illustrated in Figure 9, bottom, our SR-Net will remove stripe artifacts more cleanly without loss of any vessel information than the baseline. Furthermore, Table 4 demonstrates the superiority of the proposed SR-Net in improving segmentation performances. The proposed SR-Net achieves an improvement of about 5.88, 1.89, and 1.71% in Dice, Sen, G-mean over the baseline, respectively.

In order to verify the further quality improvement on AS-OCTA images in the re-enhancing stage, we regarded the results de-striped by our SR-Net as the input low-quality imagesof all re-enhancing approaches for unified comparison. CycleGAN was adopted as the baseline and cyclic perceptual loss L_p and structure loss L_s were added to baseline, respectively to confirm the effectiveness of the both.

Visually As shown in Figure 10, top, our method will improve the continuity of vessel compared with the baseline, and our method will not generate the non-existent vessels.

Vessel segmentation Illustrated in the Table 5, we also applied COOF (61) for the vessel segmentation to verify the effectiveness of enhancement results. Our method achieves the highest Dice, Sen and G-mean when compared with the baseline, Baseline+L_p and Baseline+L_s, indicating that the proposed PS-GAN is effective to restore more vascular information. Furthermore, as illustrated in Table 5, the proposed PS-GAN can improve the segmentation performances. The proposed PS-GAN achieves Dice of 0.599, with the improvement of about 0.50, 0.84, and 0.50% over the baseline, The baseline+L_p and baseline+L_s. Meanwhile, the proposed PS-GAN achieves G-mean of 0.812, with the improvement of about 1.75, 2.27, and 1.37% over the baseline, baseline+L_p and baseline+L_s.

In addition, we also validated the effectiveness of L_p and L_s on the synthetic OCTA datasets. As shown in Table 3, PSNR and SSIM were improved by adding L_p or L_s respectively. For example, the proposed SR-Net achieves the improvement in PSNR and SSIM scores in the ROSE dataset when compared with the baseline, baseline+L_p and baseline+L_s if we added different levels of stripe noise, i.e., 3.86, 1.34, 1.07, 1.06, 0.31, and 0.42% higher in terms of PSNR and SSIM for noise level i = 3. Similarly, our method also yields better performance in the PUTH dataset when the images were corrupted by different noise levels. Furthermore, the proposed PS-GAN achieves the best performance on image enhancement compared with the baseline, baseline+L_p and baseline+L_s.