Deep learning is developing rapidly with the help of big computing (Jouppi et al., 2017) and big data (Deng et al., 2009; Sun et al., 2017; Zhou et al., 2017). Some works (Radford et al., 2019; Brown et al., 2020; Raffel et al., 2020) focus on feeding larger models with more data for better performance and generalization, while others design task-specific model structures and loss functions (Hu et al., 2019; Huang et al., 2021; Zhao et al., 2022) to improve performance on a fixed dataset. Recently, data itself has received a lot of attention. Ng et al. (2021) led the data revolution of deep learning and successfully organized the first “Data-Centric AI” competition. The competition aims to improve data quality and develop data optimization pipelines, such as label correction, data synthesis, and data augmentation (Motamedi et al., 2021). Competitors mine data potential instead of optimizing model structure to improve performance. Northcutt et al. (2021) found that if the error rate of test labels only increases by 6%, ResNet18 outperforms ResNet-50 on ImageNet (Deng et al., 2009). To improve data quality and accurately evaluate models, there is an urgent need to develop label correction algorithms. In weak supervision and semi-supervision (Li et al., 2020; Zheng et al., 2021), pseudo label correction is usually implemented due to the lack of supervision from real labels. Zheng et al. (2021) correct the noisy labels by using a meta network for image recognition and text classification. For supervised learning, bad data can be discarded by data preprocessing, but bad labels seem inevitable in large-scale datasets. In crowdsourcing (Bhadra and Hein, 2015; Nicholson et al., 2016), an image is annotated by multiple people to improve the accuracy of classification task (Nicholson et al., 2015; Kremer et al., 2018; Guo et al., 2019). Guo et al. (2019) trained a model by using a small amount of data and design a label completion method to generate labels (negative or positive) for the mostly unlabeled data. However, label correction in these tasks is significantly different from correcting object contours. In this paper, to eliminate edge location errors and inter-annotator differences in manual annotation, we propose an label correction method based on annotation points for edge detection and image segmentation. Besides, we compare our LLPC with conditional random fields (CRF) (Sutton et al., 2012), which is popular as post-processing for other segmentation methods (Chen et al., 2017; Sun et al., 2020; Fan et al., 2021a; Lu et al., 2021; Ma et al., 2022; Zhang et al., 2022). Dense CRF (Krähenbühl and Koltun, 2011) improves the labeling accuracy by optimizing energy function based on coarse segmentation images, while our LLPC is a label correction method based on annotation points, which are two different technical routes of label correction for image segmentation. More discussion in Section 5.3.

Currently, cervical cell datasets include ISBI 2015 challenge dataset (Lu et al., 2015), Shenzhen University dataset (Song et al., 2016), and Beihang University dataset (Wan et al., 2019). Supervised deep learning based methods require large amounts of data with accurate annotations. However, the only public ISBI dataset (Lu et al., 2015) has a small amount of data and simple image types, which are difficult to train deep neural networks. In this paper, we construct a largest high-accuracy cervical cell edge detection dataset based on our label correction method. Our CCEDD contains overlapping cervical cell masses in a variety of complex backgrounds and high-precision corrected labels, which are sufficient in quantity and richness to train various deep learning models.

Although the annotations of cervical cell images are provided by professional cytologists, due to human error, the label points usually deviate from the pixel gradient peaks. To solve this problem, we design a gradient-guided point correction (GPC) method based on gradient guidance. We correct the label points only in the strong gradient region to eliminate human error, while preserving the original label points in the weak gradient region to retain the correct high-level semantics in human annotations. Our point correction consists of three steps as follows:

The processing object of our LLPC is a set of label points ({xsi}) corresponding to a closed contour. For an original label point xsi, we select candidate points along the normal direction of label edge, as shown in Figure 2A. These points constitute a candidate point set Ωxsi, and xmaxi is the point with the largest gradient in Ωxsi. We move xsi to the position of xmaxi to obtain the corrected label point xci.

where

xsji is a candidate point in Ωxsi. We judge whether a point xsi is in strong gradient regions through Δ. If Δ > 0, the point will be corrected; otherwise, it will not be moved. In this way, when the radius (r) of Ωxsi is larger, our method can correct larger annotation errors. However, this will increase the correction error of label points due to image noise and interference from adjacent edges. To balance the contradiction, the gradient value of the candidate point g(xsji) is weighted by ω_j, which allows setting a larger radius to correct larger annotation errors. We compute the weight as

where

K(x, h) is a weighted kernel function with bandwidth h. κ(x) is a Gaussian function with zero mean and one variance. After point correction, {xsi}→{xci}.

The edge generated directly from the point set {xci} is not smooth due to the errors in point correction process (see Section 5.4). To eliminate the errors, we fit multiple curve segments and stitch them together. In the annotation process of manually drawing cell contours, the annotators perform dense point annotations near large curvatures, and sparse annotations near small curvatures to accurately and quickly outline cell contours. Since the existence of large intervals is not conducive to curve fitting, we perform linear point interpolation (PI) on these intervals before curve fitting.

In Section 3.2, we stitch multi-segment curves to obtain a closed cell curve, and then sample the curve to generate a discrete edge. In fact, there is no smoothness at the splice nodes. To generate a smooth closed discrete edge, we design a local point smoothing (LPS) method without curves splicing and sampling. As shown in Figure 4A, we insert more points in large intervals (gap = 1). As shown in Figure 4B, we only correct the center point of Φkr by fitting a curve (C_k). By shifting the local coordinate system by one step (s = 1), each point in {xIj} will be corrected by fitting a curve. These correction points constitute a discrete edge C_d. Because no curves are spliced, the generated edge is smooth at each point. The pipeline of our LLPC is shown in Algorithm 1.

In Section 3.1, we set the parameters r = 15, λ_t = 4 and h₁ = r/2. In Section 3.2, we set n_g = 14. r_f = ⌊(n_g − 1)/2⌋. h₂ = r_f/2. When gap = 1 and s = 1, the Section 3.3 is a special case of the Section 3.2. See Section 5.4 for more discussion of parameter selection.

We compare our CCEDD with other cervical cytology datasets in Table 1. Our dataset was collected from Liaoning Cancer Hospital & Institute between 2016 and 2017. We capture digital images with a Nikon ELIPSE Ci slide scanner, SmartV350D lens and a 3-megapixel digital camera. For patients with negative and positive cervical cancer, the optical magnification is 100× and 400×, respectively. All of the cases are anonymized. All processes of our research (image acquisition and processing, etc.) follow ethical principles. Our CCEDD dataset includes 686 cervical images with a size of 2,048×1,536 pixels (Table 2). Six expert cytologists outline the closed contours of the cytoplasm and nucleus in cervical cytological images by an annotation software (labelme; Wada, 2016).

We randomly shuffle our dataset and split it into training, validation and test sets. To ensure test reliability, we set this ratio to 6:1:3. To be able to train various complex neural networks on a GPU, we crop a large-size image into small-size images. If an image is cut as shown in Figure 5A, it will result in incomplete edge at the cut boundary. To maximize data utilization efficiency, we move the cutting grid, as shown in Figures 5B–D. After label correction, we cut an image with a size of 2,048×1,536 into 49 image patches with a size of 512×384 pixels.

We show the visual comparison results on our CCEDD in Figure 6. The quantitative comparison is shown in Table 4 and Figure 1B. These results have important guiding implications for accurate edge detection of overlapping cervical cells. We analyze several factors affecting the performance of overlapping edge detection.

In our LLPC, the position of label points is locally corrected to the pixel gradient peak. As shown in Figures 1A, 8B, Our LLPC can generate more accurate edge labels. Besides, we can easily generate corrected masks from corrected points in the labelme software (Wada, 2016). Compared with the original mask in Figure 8C, our corrected mask has higher edge localization accuracy and smoother edges, as shown in Figure 8D.

We train multiple networks using original label and corrected label. The quantitative comparison results is shown in Table 3 and Figure 1B. Compared with the original label, using the corrected label to train multiple networks can significantly improve AP (30–40%), which verifies the effectiveness of our label correction method. Table 4 shows that the performance improvement comes from two aspects. First, our corrected label can improve the evaluation accuracy in testing (0.541 → 0.588). Second, using our corrected label to train network can improve the accuracy of overlapping edge detection in training (0.588 → 0.755), as shown in Figure 9.

In Figures 10, 11, we compare our LLPC with active contours (Chan and Vese, 2001) and dense CRF (Krähenbühl and Koltun, 2011). We observed that active contours (Chan and Vese, 2001) is refinement failure of nucleus contours in Figure 10F, and dense CRF (Krähenbühl and Koltun, 2011) fails due to complex overlapping cell contours in Figure 11C. Since active contours (Chan and Vese, 2001) and dense CRF (Krähenbühl and Koltun, 2011) are global iterative optimization methods based on segmented images, which are uncontrollable for label correction of object contours and ultimately lead to these failed results. Our LLPC is the local label point correction without iterative optimization. Therefore, the correction error of our LLPC is controllable and the error in one place does not spread to other places, which is crucial for robust label correction. Besides, dense CRF (Krähenbühl and Koltun, 2011) is nonplussed over overlapping instance segmentation refinement, while our LLPC corrects label based on annotation point and can handle overlapping label correction, as shown in Figure 11E.

Our label correction method can correct a closed contour by correcting the position of label points, which does not require additional prior assumptions (e.g., contour shape, object size). We annotated several images in the PASCAL VOC dataset (Everingham et al., 2010) with labelme (Wada, 2016) and corrected the label (r = 7, λ_t = 4, and n_g = 9). As shown in Figure 15, our label correction method can generate more accurate object contours, which demonstrates the feasibility of our label correction method for natural images.

Overlapping edge detection of cervical cell is a challenging task due to the presence of strong and weak gradient edges. For edges with strong gradients, it only requires low-level detail features. For edges with weak gradients in overlapping region, it may require high-level semantics to reason contours and connect edges based on the context in strong gradient regions. While Unet++ (Zhou et al., 2019) achieves the best results on our CCEDD, there is no difference in the detection of these two different types of edges. Designing new network structures and loss functions for overlapping edge detection may be a way to further address this challenge.