Harbin (45°42′–46°28′N, 126°29′–130°01′E) is located in the central-southern part of Heilongjiang Province and falls within the warm temperate continental monsoon climate zone. With an urban tree coverage rate of approximately 40%, the city experiences extremely low temperatures in winter, frequently accompanied by severe cold conditions (Jin et al., 2019). The vegetation primarily consists of deciduous and coniferous tree species, with dominant varieties including elm, willow, pine, poplar, and birch.

The theoretical basis for species classification integrates geometric characteristics and spectral analysis. Ulmus (elm) exhibits an irregularly rounded crown with relatively uniform foliage distribution. Salix (willow) is characterized by pendulous branches and an oblong or elliptical crown with a loosely defined margin. Pinus (pine) typically presents a conical or umbrella-shaped crown featuring dense foliage and a serrated silhouette, a key identifier of coniferous traits. Populus (poplar) displays a tall, columnar crown with a flattened apex. Betula (birch) is distinguished by its whitish bark and an ovate crown exhibiting a grayish-green hue.

Manual classification relying solely on geometric features is susceptible to subjective bias and inconsistency. Therefore, spectral feature analysis was performed using TIF-format remote sensing imagery, incorporating the blue, green, red, and near-infrared channels along with the Normalized Difference Vegetation Index (NDVI). NDVI is an indicator for assessing vegetation vigor and coverage, calculated from the reflectance in the near-infrared and red bands of remote sensing images (Rezatofighi et al., 2019). Its values range from -1 to 1, and an NDVI value greater than 0.6 typically indicates dense and healthy tree cover. The calculation formula is shown in Equation 1.

Here, NIR represents the reflectance in the near-infrared band, and Red denotes the reflectance in the red band.

The spectral reflectance characteristics and vegetation indices of the five tree species are summarized in Table 1. The wavelength ranges for the spectral bands are defined as follows: blue (450–500 nm), green (500–600 nm), red (600–700 nm), and near-infrared (700–900 nm).

The remote sensing imagery for the Harbin area, Heilongjiang Province, China, was acquired from Google Earth Engine and exported in GeoTIFF format to constitute the dataset. The images were captured on August 10, 2022, with a sensor altitude of 280 meters above sea level. Using the Python Pillow library, the TIF images were converted to PNG format. The final dataset comprises 2,500 images, each with a pixel size of 640 × 640.

The image data were annotated using Labelme. This tool saves the annotation information in JSON format, which is structured, easy to store, and machine-readable. The entire JSON file constitutes a dictionary-like structure, where various metadata elements are stored in key-value pairs. For instance, the “imageWidth” and “imageHeight” fields represent the width and height of the image in pixels, respectively.

To ensure the reliability of the dataset, we conducted a rigorous field verification campaign in August 2022. We randomly selected 500 samples from the annotated dataset, covering all five tree species, and verified their ground truth categories using a handheld GPS device (accuracy ±1m) and field photography. This on-site validation confirmed that our expert visual interpretation achieved an accuracy of over 96%, with minor corrections applied primarily to distinguish between young Populus and Salix trees in shaded areas.

To enhance model performance and generalization capability, ensuring its robustness in complex and varied image recognition tasks, data augmentation was performed through image rotation and mirror flipping. This process simulates the appearance of images from different perspectives. Mirror flipping includes both horizontal and vertical flipping, which increases data diversity and exposes the model to a wider range of directional variations in image samples.

Since both rotation and flipping significantly alter the position and orientation of target objects in the image, it is essential to update the corresponding annotation information promptly and accurately. The bounding boxes must be adjusted accordingly to ensure precise alignment with the transformed objects. An example of the data preprocessing procedure is illustrated in Figure 1. To ensure the integrity of model evaluation and prevent data leakage, the original dataset of 2,500 images was first split into training, validation, and test sets in a ratio of 8:1:1. Subsequently, data augmentation (rotation and flipping) was applied exclusively to the training set, expanding it to 15,000 images, while the validation and test sets remained unaugmented to represent real-world scenarios.

The YOLOv11n architecture (Figure 2) primarily consists of three core components: a Backbone, a Neck, and a Head.

The backbone network extracts hierarchical feature representations from the input image, and this process begins with a series of Convolution-Batch Normalization-SiLU (CBS) modules. Within these modules, the convolutional layer performs feature extraction, utilizing kernels of varying sizes and strides to capture local information at different scales. The Batch Normalization (BN) layer then normalizes the output of the convolution, accelerating model convergence and enhancing its generalization ability. Finally, the SiLU activation function introduces non-linear transformations, which augment the model’s expressive capacity and enable it to learn more complex patterns.

The Residual Connections facilitate the learning of residual information via skip connections, which mitigates the vanishing gradient problem in deep network training and enables the network to excavate more profound features. Furthermore, Group Convolution operates independently across multiple branches, significantly reducing both parameter count and computational overhead. This synergistic combination allows the subsequent C3k2 module to efficiently extract deep image features while minimizing computational cost and model parameters. Consequently, the design not only maintains detection accuracy but also enhances operational efficiency in resource-constrained environments.

The backbone network further incorporates a Spatial Pyramid Pooling-Fast (SPPF) module and a C2PSA module. The SPPF module is designed to enhance feature extraction capabilities. The input feature map first undergoes a preliminary transformation via a convolutional layer. It is then fed into three parallel max-pooling layers, which perform down-sampling at different scales to generate multi-scale features. These multi-scale features, along with the original input features, are concatenated, effectively integrating fine-grained details with broader contextual information. The fused features are subsequently refined through another convolutional layer to better suit the requirements of the object detection task.

The C2PSA module is constructed based on the Squeeze-and-Excitation (SE) module and the Pyramid Split Attention (PSA) module. The input feature map is first processed by a convolutional layer for initial transformation. A split operation then distributes the features into multiple branches, each of which is processed by a PSA sub-module. A structural diagram of the C2PSA module is presented in Figure 3.

The C2PSA module employs a multi-step, multi-dimensional feature processing strategy to more effectively capture subtle inter-class distinctions. Specifically, feature outputs from multiple parallel PSA submodules are first fused via a concatenation layer. The combined features then pass through a Softmax operation and are further aggregated through feature concatenation (x_concat), thereby enhancing the modeling of spatial dependencies. Subsequently, the fused features are refined and integrated by a convolutional layer. This structured approach enables the C2PSA module to sensitively discern fine-grained differences among target categories.

The primary function of the neck network is to integrate the hierarchical feature maps generated by the backbone, thereby constructing a feature pyramid rich in semantic information and multi-scale representations. Through iterative upsampling and concatenation operations, it progressively enhances the discriminative power and robustness of the features, ultimately providing superior input for the detection head.

The head network takes the fused feature maps from the neck as input. It then processes them through a series of dedicated Detect modules to produce the final detection outcomes. Each Detect module typically consists of convolutional layers for further feature refinement and transformation, followed by a prediction layer that estimates both the class probabilities and spatial coordinates of the objects.

The head network of YOLOv11n generates detection outputs across three distinct scales, with dimensions of 80×80, 40×40, and 20×20 (each with 255 channels). This multi-scale design enables the model to effectively detect objects of varying sizes and categories within an image. By leveraging both fine-grained and high-level semantic information, this approach significantly enhances the comprehensiveness and accuracy of object detection.

To enhance feature representation, particularly for small objects, we introduce several key modifications to the YOLOv11n architecture. First, the Convolutional Block Attention Module (CBAM) is hierarchically integrated into the backbone network at layers 5, 7, and 9. This lightweight module sequentially infers attention maps along both the channel and spatial dimensions, allowing the model to adaptively emphasize informative features while suppressing less useful ones. The refined C3k2-CBAM module is thereby empowered to better excavate the latent features of small objects, effectively mitigating the information loss caused by low pixel counts. Second, we replace the original EIoU loss function with the Normalized Wasserstein Distance (NWD) loss. This substitution shifts the bounding box regression loss to be more sensitive to small objects, leading to an overall enhancement in model performance.

To address the limitations of the standard CBS (Conv-BN-SiLU) module in detecting small targets within remote sensing imagery, we strategically replaced it with a more efficient GBS module at layers 4, 6, 8, and 20 of YOLOv11n. While the CBS module serves as a common building block in the architecture, it is not optimal for small objects in complex scenes. The GBS module overcomes this by optimizing kernel configurations, weight distributions, and activation functions. This redesign significantly reduces the computational overhead and parameter footprint, enabling the model to operate efficiently even under resource constraints while maintaining a focus on small targets.

To enhance the model’s ability to capture geometric features of urban trees, we strategically integrated the Deformable Convolution Network v3 (DCNv3) module at two critical points in the YOLOv11n architecture. The DCNv3 module improves upon standard convolution by introducing learnable offsets, allowing the convolutional kernel to adaptively sample features from irregular shapes and positions, thereby capturing more comprehensive object characteristics. First, within the backbone network at layer 11, we optimized the existing C2PSA module by incorporating DCNv3, resulting in the C2PSA-D3 module. This integration maximizes the benefits of deformable convolution without a substantial increase in computational overhead. Second, we enhanced the model’s detection head by replacing standard convolutions with DCNv3, forming the Detect-D3 module. This upgrade provides the small-object detection head with richer feature information. Collectively, these replacements significantly boost the model’s capacity to discern tree edges and shapes, leading to superior detection accuracy in challenging scenarios, such as complex backgrounds and occluded conditions. The overall architecture of the proposed YOLO-CNGD model is depicted in Figure 4.

The outcomes of the detection experiments can be categorized into four fundamental cases: true positive (TP), false positive (FP), true negative (TN), and false negative (FN).

1. Precision (P) is defined as the ratio of correctly detected targets to the total number of targets detected by the model Equation 2. In the given context, a True Positive (TP) represents an instance where a tree is correctly identified as one of the five species, while a False Positive (FP) denotes an instance that is incorrectly classified as one of the five species.

2. Recall (R) represents the percentage of actual targets that are correctly identified by the model Equation 3. Here, a False Negative (FN) represents a case where the target belongs to one of the five tree species, but the model fails to detect it.

3. The F1-score is the harmonic mean of Precision and Recall, providing a single metric that balances the trade-off between these two values Equation 4.

4. Average Precision (AP) is defined as the area under the Precision-Recall (P-R) curve, which plots Precision (y-axis) against Recall (x-axis). A robust model maintains high precision as recall increases, resulting in a larger area under the curve and thus a higher AP value. Typically, an Intersection over Union (IoU) threshold of 0.5 is used for this evaluation. A higher AP value indicates better detection performance. The formulas for computing AP and mean Average Precision (mAP) are given in Equations 5 and 6, respectively.

All experiments were conducted on a workstation equipped with an Intel Core i9-12900K CPU and an NVIDIA GeForce RTX 3090 GPU (24GB). The operating system was Ubuntu 20.04, and the deep learning framework used was PyTorch 1.12.1 with CUDA 11.3. The model was trained for 200 epochs using the SGD optimizer. The initial learning rate was set to 0.01 with a momentum of 0.937 and weight decay of 0.0005. The batch size was set to 32. We utilized a cosine annealing strategy for learning rate decay. The dataset of 15,000 images was randomly partitioned into training, validation, and test sets in a ratio of 8:1:1.

Regarding model complexity and hardware demands, the YOLO-CNGD model has 3.05 million parameters and requires 8.2 GFLOPs for a single 640×640 image inference. The total training process for 200 epochs on the NVIDIA RTX 3090 lasted approximately 4.2 hours. During the inference phase, the model occupies only 1.1 GB of VRAM, making it highly suitable for deployment on edge computing devices with limited hardware resources.

Beyond the hold-out test set, we conducted a 5-fold cross-validation to assess the model’s stability and mitigate the impact of random data partitioning. The entire dataset of 15,000 images was randomly divided into 5 equal-sized folds. In each iteration, four folds were combined and then split into training and validation subsets, while the remaining fold was used as the test set. This process was repeated five times with each fold serving as the test set once. The final performance metrics reported are the mean ± standard deviation calculated across all five test folds. A paired t-test was conducted between YOLO-CNGD and the baseline YOLOv11n, yielding a p-value < 0.05, which confirms that the performance gains are statistically significant. This approach provides a more reliable estimate of model generalizability and allows us to compute confidence intervals for our results.

To benchmark the performance of our model, we trained and evaluated several state-of-the-art object detection algorithms, including Faster R-CNN, SSD, RetinaNet, YOLOv3, YOLOv5, YOLOv7, YOLOv8, and YOLOv11n, on the urban tree remote sensing dataset. A comprehensive comparison was conducted using metrics such as model size (number of parameters), detection accuracy (Precision, Recall, F1-score), and comprehensive detection performance (mAP@0.5, mAP@0.5:0.95). The quantitative results are summarized in Table 2, and the corresponding performance curves are visualized in Figure 8.

In addition to detection accuracy, we evaluated the inference speed, a crucial metric for real-time monitoring. On the NVIDIA GeForce RTX 3090 GPU, the proposed YOLO-CNGD achieved an inference speed of 112 FPS (Frames Per Second) with an average inference time of 8.9 ms per image. This performance significantly exceeds the real-time requirement (usually 30 FPS), confirming the model’s suitability for rapid large-scale urban forest surveying.

To validate the stability of the model, we performed 5-fold cross-validation. The results show that YOLO-CNGD maintains high stability, with an mAP@0.5 of 93.7% ± 0.15% (95% Confidence Interval: [93.4%, 94.0%]). This narrow confidence interval confirms that the performance improvements are statistically robust and not due to random variation in the dataset split.

The results for YOLOv11n and YOLO-CNGD are reported as mean ± standard deviation based on 5-fold cross-validation. Other comparative models reflect the best performance from standard training runs consistent with their original literature settings.

A series of ablation studies were conducted to validate the contribution of each proposed modification, with the quantitative results summarized in Table 3. The results demonstrate that our full model, which incorporates the DG (DCNv3 + GhostConv) structure, the NWD loss, and the CBAM attention mechanism, achieves significant performance gains over the baseline YOLOv11n. Specifically, it improves overall Precision by 4.1%, Recall by 4.0%, mAP@0.5 by 5.2%, and mAP@0.5:0.95 by 5.6%.

1. Parameter Count: The number of parameters is a critical indicator of model complexity, directly influencing the computational resources and time required for both training and inference. The data reveals significant disparities in parameter counts across the YOLO series. Notably, YOLOv11n, with a minimal 2.58M parameters, demonstrates superior architectural efficiency for lightweight deployment without imposing excessive hardware demands. In contrast, YOLOv3 possesses a substantially larger parameter count of 61.6M. While this high complexity suggests a greater capacity for learning rich and detailed feature representations, it consequently leads to prolonged training times and an increased risk of overfitting, which can ultimately impair the model’s generalization performance.

2. Precision: Precision serves as a core metric for evaluating the accuracy of model predictions, reflecting the correctness of positive identifications. It is calculated as the proportion of true positive instances among all samples predicted as positive, thereby directly indicating the reliability of the model in identifying target categories. Among the eight models compared, YOLOv11n achieved the highest precision of 0.907, demonstrating its strong capability in accurately determining the presence of targets with a low probability of false alarms. In contrast, Faster R-CNN attained a notably lower precision of 0.637, suggesting that it may produce a higher number of false positives during detection.

To further analyze the classification errors, we examined the confusion matrix of the proposed model,as presented in Figure 9. The results reveal that the majority of misclassifications occur between Salix (Willow) and Populus (Poplar). This confusion is primarily attributed to two factors: first, the high spectral similarity of their leaves in the visible bands makes them difficult to distinguish based on color alone; second, in areas with dense vegetation, shadows often obscure the distinctive crown shapes (e.g., the pendulous branches of willows), leading to edge prediction errors. In contrast, Pinus (Pine) exhibits the least confusion with other species due to its unique coniferous texture and needle-like foliage.

The matrix validates the model’s performance on the validation set (N = 1500). The rows represent the actual labels, and the columns represent the predicted labels. The diagonal elements indicate correct classifications. The orange-highlighted cells quantitatively demonstrate the primary spectral confusion between Salix (Willow) and Populus (Poplar),

3. Recall: Recall measures a model’s ability to detect all positive instances, defined as the proportion of actual positives correctly identified. YOLOv11n and YOLOv8 achieved outstanding recall scores of 0.872 and 0.870, respectively, indicating their strong capability in capturing target objects and minimizing missed detections. In contrast, Faster R-CNN attained a recall of only 0.608, significantly lower than the top-performing models, suggesting a tendency to overlook a considerable number of actual positive instances during detection.

4. F1-score: As the harmonic mean of precision and recall, the F1-score provides a balanced assessment of a model’s prediction accuracy and detection completeness, offering a comprehensive evaluation of overall performance. YOLOv11n achieved the highest F1-score of 0.894, indicating an optimal balance between precision and recall. This result demonstrates its capability to maintain high prediction accuracy while effectively detecting the majority of target instances. In contrast, Faster R-CNN obtained an F1-score of only 0.622, suggesting a higher incidence of missed detections and highlighting the need for further optimization in practical applications.

5. mAP@0.5 and mAP@0.5:0.95: mAP@0.5 represents the mean average precision calculated at an IoU threshold of 0.5, reflecting the model’s detection performance under a relatively lenient bounding box matching criterion. This metric emphasizes the model’s preliminary capability to identify target presence. YOLOv11n achieved an mAP@0.5 of 0.885, indicating its strong performance in accurately detecting targets under relaxed localization requirements. This makes it particularly suitable for applications where rapid identification of approximate target locations is prioritized over precise boundary delineation. In contrast, mAP@0.5:0.95 is computed across multiple IoU thresholds (from 0.5 to 0.95 with a step size of 0.05), imposing stricter localization accuracy demands and providing a more comprehensive assessment of the model’s detection robustness in challenging scenarios. YOLOv11n also attained the highest score of 0.571 in this rigorous metric, demonstrating its consistent ability to maintain high detection precision across varying degrees of target overlap.

In summary, while Faster R-CNN, SSD, and RetinaNet exhibit relatively weaker performance on certain metrics, the YOLO series demonstrates prominent results across multiple aspects. Among them, YOLOv11n achieves outstanding detection accuracy with a notably low parameter count of only 2.58M. It performs excellently in terms of precision (0.907), recall (0.872), F1-score (0.894), as well as mAP@0.5 (0.885) and mAP@0.5:0.95 (0.571). This model strikes an effective balance between lightweight design and reliable detection performance, making it highly suitable for real-world applications that require efficient deployment under limited computational resources without compromising on accuracy. YOLOv11n thus represents a cost-effective and competitive solution in the field of urban tree detection. To ensure the reliability of the improvements, we repeated the training process multiple times. The proposed YOLO-CNGD consistently outperformed the baseline, and the improvement in mAP@0.5 (5.2%) is considered statistically significant given the stability of the training curves.

We further examined the confusion matrix to analyze specific misclassifications, revealing that the primary inter-class confusion occurs between Salix (Willow) and Populus (Poplar). As visualized in Figures 10A–C, these failure cases are mainly attributed to heavy shadows and dense canopy occlusion in complex urban environments, which render the spectral signatures of these two species indistinguishable. Under such conditions, the characteristic drooping branches of willows are often obscured, leading to the observed misclassifications.”

To extract more discriminative features from data and enhance the performance of the YOLOv11n model in object detection tasks, we conduct a series of ablation studies. These experiments are designed to systematically investigate the individual contributions of various components and mechanisms to the overall model performance.

The accurate, automated classification of urban tree species achieved by YOLO-CNGD transcends mere technical performance, offering tangible value for urban forestry practice and ecological planning. The transition from manual interpretation to AI-driven mapping enables scalable, data-informed decision-making across several key domains.

Despite the promising results, this study has several limitations that need to be addressed in future research. First, regarding dataset diversity, the current dataset is limited to Harbin, China, and comprises images captured on a single date (August 10, 2022). Consequently, the model’s generalization capability across different climatic zones and phenological seasons (e.g., autumn leaf coloration or winter defoliation) remains to be validated. Future work will expand the dataset to include multi-temporal and multi-regional imagery to test the model’s transferability. Second, regarding failure cases, although the model handles small objects well, performance drops in scenarios with dense canopy occlusion or heavy shadows cast by high-rise buildings. In these cases, the spectral features of the shadowed trees are distorted, leading to misclassifications between species with similar crown shapes (e.g., Willow and Poplar). Third, regarding taxonomic scope, this study focused on five dominant tree species. While sufficient for Harbin’s urban core, expanding the class categories to include shrubs and other rare species would enhance the tool’s applicability for broader biodiversity surveys. Future work will explore fusing multi-spectral indices (e.g., enhancing red-edge bands) or temporal data to disambiguate spectrally similar species under varying illumination.