To find an algorithm suitable to detect cold seep epifauna, we considered two-stage (Faster R-CNN base model and our improved model) and single-stage [including SSD300 (Wang et al., 2017) and YOLOv3] detection algorithms to test the same epifauna dataset. The results showed that the two-stage detection model performed better for the average precision (Table 2). Thus, it was more suitable to detect and count cold seep epifauna. However, two-stage detection models are not suitable for real-time target detection because of their relatively lower speeds over single-stage detection. Compared with the original Faster R-CNN structure and other algorithms, the improved algorithm developed in this paper has a higher detection accuracy. Although the detection time increases, the improvement was acceptable relative to the performance. The Fast R-CNN + FPN model was chosen for the recognition of cold seep substrates and epifauna.

The pyramid form of the CNN used in the FPN effectively generates multi-scale feature expressions under a single image view. The FPN structure was designed with a top-down structure while lateral connections fused the shallow high-resolution feature map with the deep feature map having rich semantic information. Embedding the FPN structure in the Faster R-CNN allowed quickly building a feature pyramid with strong semantic information at all scales that could significantly improve the detection capability of the network for small-scale targets (Lin et al., 2017). The FPN structure is showed in Figure 2, where bottom-up usually refers to the forward computing process of the backbone; top-down uses nearest-neighbor interpolation for up-sampling on the higher-level feature map which could retain the semantic information of the feature map to the maximum extent; and the lateral connection adjusts the feature dimension with a 1 × 1 convolution to ensure consistent corresponding feature layer dimensions between the bottom-up process and the top-down processes. Then, the 3 × 3 convolution operation was aimed at eliminating the blending effect generated in the feature fusion process.

Embedding the FPN structure in the Faster R-CNN fused the deep and shallow features, which strengthened the feature expression ability of the network and effectively improved its accuracy for small target detection. In addition, ResNet50 (Szegedy et al., 2017) was used to replace the VGG16 (Simonyan and Zisserman, 2014) as the backbone feature extraction network of the Faster R-CNN to further improve the network capabilities.

The network architecture of the improved Faster R-CNN algorithm is shown in Figure 3. The feature fusion maps for different levels as extracted by the FPN were separately fed into the region proposal network (RPN) network. This network generated anchor frames with different aspect ratios based on the different feature map scale sizes and selected the corresponding detection layer for the target. The region of interest (ROI) pooled the corresponding region into a fixed-size feature vector in the feature map based on the position coordinates of the candidate regions.

We set a binary label for each anchor to judge whether it was a target or not to train the RPN. We set a binary label for each anchor to judge whether it was a target or not: we set a positive label for all anchors whose intersection over union (IOU) was greater than or equal to 0.7. We then set a negative labor for all anchors with IOU less than 0.3 for the real box. The loss function of a single image is defined as:

where i is the index of the candidate box in a batch of data, P_i is the prediction probability of the i th candidate box as the target. If the candidate box is positive, the real label P_i = 1; otherwise, Pi*=1. The λ is the normalized weight with λ=10 set in the experiment, and t_i is the four parameterized coordinate vectors of the prediction box and was the real box vector related to positive samples. The N_cls is the batch size in the training process, N_reg is the number of candidate boxes, and L_cls is the binary logarithm loss and defined as:

The L_reg is the regression loss function and is defined as:

where the smooth function is defined as:

where x is the error of the border prediction and σ is used to control the smooth area with a values of 3 in the experiment.

The complexity of cold seep substrates and the problems of color bias, dark light and blur in underwater images made it difficult to distinguish all substrate categories using a single classifier. This paper adopted an integration strategy to improve the accuracy of recognition and classification accuracies. In the classification, several base classifiers responsible for distinguishing part of the data were connected in series. The undifferentiated data went to the subsequent base classifier to achieve the optimal results in the iteration.

The associated substrate classification model framework is shown in Figure 4. The first classification model was the FPN embedded in the Faster R-CNN to classify and count the biota. The images that did not satisfy the current classification conditions were passed to the next CNN model, which classified the zones of shell debris and exposed authigenic carbonates. Then, the images that did not satisfy these classification conditions continued to the VGG16 model for binary classification which was responsible for identifying the reduced sediment and muddy bottom zones.

The first base of the integrated classifier in the figure consists of the improved Faster R-CNN and FPN from the section “Faster R-CNN + FPN Algorithm for Epifauna Detection.” The backbone network used for feature extraction in the model, which could greatly improve the feature extraction capability was replaced by VGG16 with ResNet50 as fused with the FPN. The first base classifier was mainly responsible for biota classification. The FPN was embedded into the Faster RCNN to reduce the redundant multi-resolution feature map detection from the original structure based on the realization of feature fusion. Nearest-neighbor upsampling was used to fuse the shallow location and deep semantic information of the network. With only a small increase in computational costs, the detection accuracy of the network for multi-scale targets was greatly improved. Classification and identification allowed counting the epifauna. Then, the zones of the lobster cluster, mussel bed and biological mixing were distinguished using the substrate classification rules.

The second base classifier consisted of the CNN which was aimed primarily at the recognition and classification of the shell debris and carbonate rock zones. This structure could reduce the amount of memory occupied by the deep network, effectively reduce the number of network parameters, and alleviate the overfitting problem of the model. As a supervised multilayer learning neural network, the implicit convolutional and pooling layers were the core modules to realize feature extraction in the CNN. The CNN improved the accuracy of the network through frequent iterative training by minimizing the loss function using the gradient descent method and adjusting the weight parameters in the network layer by layer in reverse (Albawi et al., 2017). The low hidden layer of the CNN consisted of alternating convolutional and maximum pooling layers, and the high level was a fully connected layer corresponding to the implicit layer and logistic regression classifier of a traditional multilayer perceptron. The input of the first fully-connected layer was a feature map obtained via feature extraction from the convolutional and subsampling layers. The final output layer was a classifier for the input image using logistic regression, Softmax regression, or support vector machines. In this paper, a Softmax nonlinear classifier was used to identify and classify the zones of shell debris and exposed authigenic carbonates.

The third base classifier was aimed at the recognition and classification of reduced sediments and muddy bottoms. The VGG16 was composed of five convolutional layers, three fully-connected layers, and an output layer. The layers were separated by a maximum pooling layer and a ReLU activation function for all implicit activation units. On one hand, the convolutional layers with larger kernels could reduce the parameters. On the other hand, these were equivalent to more nonlinear mappings which could increase the expressive power of the network (Simonyan and Zisserman, 2014). However, the nonlinear classifier Softmax was still used for identification and classification.

The Faster R-CNN can be divided into two parts: the RPN and Fast R-CNN network. The former is a recommendation algorithm for candidate boxes (proposal), and the latter is based on the position of the box where the associated categories of objects are calculated. The RPN was trained first, and the Fast R-CNN was trained with the output (proposal) of the RPN. Fast R-CNN was fine-tuned and used to initialize the RPN parameters, which gives an iterative cycle.

The experimental data were 630 images (randomly selected from the upper part of the Mosaic), and the MRLabeler v1.4 software was used for annotation. The model could achieve the best effect when the learning rate was 0.001, the weight decay was set to 0.0004, the anchor scaling multiple was set to (8, 16, 32), and the scaling ratio was (0.5, 1, 2). To avoid overfitting, the impulse gradient was used with impulse of 0.9. The entire experiment was conducted on a Linux CentOS 7 server, and two NVIDIA GeForce GTX 1080ti graphics processing units (GPUs) with 12GB of memory were synchronized. The in-depth learning framework was implemented with tensorflow version 1.11.0 on the GPU. During training, the positive samples were labeled with IOU values greater than 0.7 and negative samples for those less than 0.3. The optimal model was saved after training and used to count the test data set.

The results of each experiment were evaluated using the Recall, Precision and Accuracy (AP). The Recall and Precision are defined as:

where the definitions of TP, TN, FP, and FN as shown in Table 3 denotes true-positive, true-negative, false-positive, and false-negative identifications, respectively. The Precision denotes the proportion of all targets predicted to be correct and the Recall denotes the proportion of targets identified by the correct localization to the total number of targets.

The MRLabeler v1.4 software was used to label the 1,501 images (randomly selected from the upper part of the Mosaic) in the experimental data set. The specific classification of substrates is shown in Table 1. A total of 303 images were obtained from the biota data set, including 15 images of lobster clusters, 240 images of mussel beds, and 48 images of biological mixing areas. There were 1,198 images of abiotic areas, including 284 of shell debris areas, 107 of exposed authigenic carbonates, 19 of reduced sediments and 788 of muddy bottoms.

The training parameters were set so the first base classifier was the same as the FPN + Faster R -CNN model in the section “Faster R-CNN + FPN Algorithm for Epifauna Detection.” In the second and third base classifiers, the convolution kernel was 3 × 3 and the step size was 1. To ensure optimal performance of each classifier, every detector was trained separately.