Given the scarcity and inherent limitations of currently available public rice pest datasets, this study aimed to construct a more challenging dataset that closely reflects real-world agricultural scenarios. To achieve this, images from two large public datasets were integrated and curated: IP102 (Wu et al., 2019) and Pest_V2 (Quach et al., 2024). A dual-construction strategy was employed to enhance the dataset’s specificity and complexity.

First, acknowledging the significant morphological differences of rice pests across various life cycle stages, we subdivided the images within the Pest_V2 dataset. Different growth stages of the same pest (e.g., larva, adult) were annotated as distinct and independent categories. Second, to further increase the challenge posed by background complexity, pest images from the IP102 dataset that were both relevant to rice and featured more intricate backgrounds were manually selected and extracted to supplement the new dataset.

Through the aforementioned process, a customized dataset named RicePest_12 was ultimately constructed. This dataset comprises a total of 2,807 high-quality images, covering 12 common categories of rice pests, such as the Asiatic Rice Borer, Brown Plant Hopper, and Paddy Stem Maggot (as shown in Figure 1). The creation of this dataset provides a high-quality and challenging experimental foundation for our research, ensuring the reliability and validity of the model evaluation.

Although the constructed RicePest_12 dataset is of high quality and presents significant challenges, its total volume of 2,807 images is still relatively insufficient. Furthermore, the dataset exhibits a notable disparity in the number of samples among different categories, presenting a typical long-tail distribution. This data imbalance issue can cause the model to develop a bias towards the majority classes during training, thereby weakening its generalization ability for rare categories and potentially leading to overfitting.

To address the aforementioned issues, this study employed a comprehensive data augmentation strategy to expand the dataset and balance the class distribution. A variety of data augmentation techniques were applied to the original images, including geometric transformations (e.g., random horizontal and vertical flips, rotation, scaling, translation, and perspective transformation) and appearance transformations (e.g., blurring). Furthermore, to enhance the model’s ability to discern complex backgrounds and reduce the false detection rate, unlabeled pure background images were strategically introduced into the training set as negative samples.

Through this augmentation and expansion, the final dataset size was increased from 2,807 to 6,374 images. The distribution of sample quantities across the various categories is detailed in Table 1. Prior to model training, this augmented dataset was randomly partitioned into training, validation, and test sets according to an 8:1:1 ratio.

This paper proposes an object detection framework named HDA-YOLO, as illustrated in Figure 2. This framework is designed to enhance the detection accuracy of multi-scale rice pests against complex backgrounds, achieving an effective integration of high precision and a lightweight structure. The overall architecture of the model is based on YOLOv8, but its core lies in the systemic, end-to-end optimization of the network’s information flow.

In the backbone network, HDA-YOLO leverages the synergistic action of an ADDS module and a MCPF module. This combination enhances the fidelity of feature extraction right from the source, providing a high-quality feature foundation for subsequent network layers. Building upon this, a novel feature fusion system was constructed: the HADF-Net. Structurally, this network employs a dense aggregation topology, utilizing cross-layer “shortcuts” to counteract information dilution. Mechanistically, it is guided by hierarchical attention, which uses an intra-scale self-attention module (ISAM) to enhance features and an inter-scale cross-attention module (ICAM) to achieve an intelligent mapping from fused features to precise predictions. Simultaneously, to further bolster the network’s multi-scale analysis capabilities during feature fusion, the C2f module in the model’s neck has been upgraded to a MSC module. The MSC module performs fine-grained analysis through parallel branches with multiple receptive fields, significantly improving the model’s adaptability to variations in target scale.

Regarding the training strategy, to enhance the model’s generalization ability to real-world field environments, images of pests under diverse lighting conditions and at various growth stages, as well as pictures with a multitude of field backgrounds, were incorporated into the dataset. This was done to increase the overall diversity and complexity of the data.

To systematically validate the effectiveness of each innovative design proposed in this study, a series of detailed ablation experiments were conducted using YOLOv8n as the baseline model. For ease of presentation and analysis, the four modules—MCPF, ADDS, MSC, and HADF-Net—are abbreviated as A, B, C, and D, respectively. The ablation study was performed using the control variable method. All experiments were conducted on our self-built rice pest dataset, with completely consistent data augmentation strategies and experimental environments maintained throughout to ensure the fairness of the comparisons.

The ablation study results in Table 3 clearly reveal how our proposed modules work synergistically to progressively enhance the model’s performance. In the initial stage, integrating module A (MCPF) enhanced the model’s ability to capture potential targets, increasing Recall by 0.7 percentage points, though this also introduced some discriminative ambiguity, causing a slight dip in mAP@50 to 87.2%. Building on this, the introduction of module B (ADDS) effectively resolved this issue. As a highly efficient downsampling module, ADDS significantly refined the features, greatly strengthening the model’s discriminative power. This led to a 1.2 percentage point leap in mAP@50 to 88.4%, with Precision also reaching its peak of 91.0%. Furthermore, the addition of module C (MSC) promoted a rebalancing between Precision and Recall. The Recall was observed to improve by 1.3 percentage points, while mAP@50 concurrently exhibited a steady increase, reaching 89.0%. Moreover, the more comprehensive metric, mAP@50-95, also demonstrated an improvement of 1.3 percentage points. Finally, with the integration of module D (HADF-Net), the complete HDA-YOLO model achieved optimal performance across all metrics. Precision returned to its peak of 91.0%, while Recall achieved a significant growth of 3.8 percentage points, pushing the F1-score to 88.2%. The mAP@50 also successfully reached 90.0%. This series of performance evolutions provides strong evidence that our four proposed modules each contribute unique and crucial performance gains, working in synergy to ultimately achieve a leap in the model’s overall performance.

To comprehensively evaluate the overall performance of the HDA-YOLO model, fair comparative experiments were conducted against several current mainstream lightweight object detection models on our self-built rice pest dataset. The comparison included models such as EfficientDet-D0, the Transformer-based RT-DETR-R18, and several representative models from the YOLO series. All models were evaluated using a unified training strategy and testing environment. The experimental results are summarized in Table 4.

As shown by the detection metrics in Table 4, the proposed HDA-YOLO model significantly outperforms all other compared models across all core metrics related to detection accuracy. Compared to the YOLOv8n baseline, our model achieves notable improvements of 2.4 and 3.8 percentage points in mAP@50 and F1-score, respectively. Furthermore, its Precision and Recall are higher by 2.8 and 4.8 percentage points. It is particularly noteworthy that when compared to the Transformer-based RT-DETR-R18, although the latter also exhibits strong performance, its computational cost (53.85 GFLOPs) and parameter count (19.51M) are far higher than our model’s. HDA-YOLO achieves superior detection accuracy while using only approximately 22% of the computational resources and 20% of the parameters, demonstrating the remarkable efficiency of deep optimization within a CNN architecture. Furthermore, when compared with YOLOv9t, which is extremely optimized for a low parameter count, HDA-YOLO leverages a moderate resource investment (3.93M vs. 2.01M parameters) in exchange for a substantial performance advantage of nearly 5 percentage points in mAP@50 (90.0% vs. 85.2%). Meanwhile, compared with YOLOv11n, which has the lowest computational complexity (6.45 GFLOPs), its Precision, Recall, and mAP@50 are 82.8%, 77.1%, and 84.3%, respectively. In contrast, HDA-YOLO improves these metrics to 91.0%, 85.6%, and 90.0% with only a slight increase in computational cost, demonstrating a more favorable trade-off between performance and efficiency.

The above experimental results demonstrate that for the task of rice pest detection, HDA-YOLO establishes a new benchmark for maximizing detection accuracy within an acceptable computational budget. By delivering significant precision gains (ranging from 2.4 to 6.0 percentage points in mAP@50) compared to various mainstream lightweight models—including the YOLO series and the Transformer-based RT-DETR-R18—the model demonstrates a superior performance-efficiency trade-off. Although there is a modest increase in GFLOPs and parameter count, HDA-YOLO’s overall complexity remains strictly within the lightweight range (12.02 GFLOPs and 3.93M parameters). This highlights its significant application potential for intelligent agricultural monitoring scenarios with high demands on detection accuracy.

To visually demonstrate the practical advantages of HDA-YOLO, Figure 8 provides a comparison of its detection results against those of several mainstream lightweight models on a set of challenging images. It can be clearly observed that HDA-YOLO, through its systematic end-to-end optimization, exhibits significant superiority across multiple complex scenarios. For the detection of small and dense objects (as shown in the fourth and sixth columns of Figure 8), HDA-YOLO accurately detects and distinguishes each individual, effectively resolving issues of missed detections, false detections, and inaccurate bounding box localization that are present in other models. This performance improvement reflects the synergistic effect of its dense fusion topology and the MSC module in combating information dilution and refining multi-scale analysis. Simultaneously, in distinguishing between the background and morphologically similar targets (as seen in the first, second, and third columns of Figure 8), HDA-YOLO demonstrates stronger robustness. It accurately identifies pests while effectively suppressing interference from similar-looking stems and leaves in the background. This enhanced capability stems from the effectiveness of the hierarchical attention mechanism and the ADDS module in optimizing feature representation. These qualitative results visually validate the synergistic effect of HDA-YOLO’s innovations in improving model accuracy while reducing false and missed detection rates, proving its significant application potential for pest monitoring in smart agriculture.

To evaluate model performance at a fine-grained level, Figure 9 once again presents a comparison of the confusion matrices between the YOLOv8 baseline model (left) and the proposed HDA-YOLO model (right) on the test set. Confusion matrices not only provide an intuitive representation of overall classification accuracy but also reveal misclassification patterns and confusion relationships among different classes.

As can be seen from the statistical results in Figure 9, the overall detection accuracy of the proposed HDA-YOLO in this study is significantly better than that of the YOLOv8 baseline model. As can be seen from the comparison, the proposed model exhibits a significant increase in the proportion of main diagonal entries (indicating correct classifications) and a noticeable reduction in off-diagonal misclassifications. This systematically validates its superior recognition robustness and classification reliability in complex scenarios. Specifically, the YOLOv8 model achieved the lowest detection accuracy of only 69% for the brown plant hopper category, followed by the asiatic rice borer larva category with an accuracy of 70%. This outcome is likely attributable to the high morphological similarity between these two pest species and the rice field background, resulting in substantial pixel-level interference in the images. In contrast, the proposed model in this study achieved detection accuracies of 76% and 74% for the two aforementioned categories, respectively. These results demonstrate a notable improvement in accuracy and highlight the model’s enhanced capability to extract pest features in complex field environments. Nevertheless, despite the significant overall improvement in accuracy, a small number of failure modes can still be observed from the confusion matrix. One type of error arises from background confusion; for example, the brown plant hopper still exhibits an approximately 24% miss-detection rate due to its strong camouflage and high visual similarity to the background. Another source of error stems from inter-class similarity, where feature overlap among morphologically similar species leads to misclassification; for instance, about 12% of brown plant hoppers are misclassified as small brown plant hoppers, and 12% of asiatic rice borers are misidentified as yellow rice borers. These results indicate that there remains room for further optimization when addressing extreme camouflage conditions and fine-grained recognition of very small targets.

To further enhance the interpretability of the improved model and provide intuitive qualitative evidence for our proposed innovations, this study focuses on the analysis of the MCPF module, the MSC module, and the HADF-Net. By employing the Grad-CAM technique (Selvaraju et al., 2019), we generate comparative feature heatmaps from before and after processing by our modules, or against the baseline modules, to improve the model’s explainability. Grad-CAM (Gradient-weighted Class Activation Mapping) is a technique designed to explain the decision-making process of convolutional neural networks. This method generates heatmaps to visualize the key image regions the model focuses on when making a decision, thereby helping to understand the reasoning behind a specific prediction.

In the heatmaps generated by Grad-CAM, deep red indicates that the model pays extremely high attention to that image region, making it a key basis for its decision. Yellow areas represent regions of lower, yet still significant, importance, while blue areas indicate that the features in that location have a minimal impact on the model’s prediction. As shown in Figure 10, the feature heatmaps output at the terminal end of the backbone network by the baseline YOLOv8n’s SPPF module and the proposed MCPF module were first compared. The heatmap from the baseline’s SPPF module exhibits distinct global context characteristics and a strong center bias. Its activation area broadly covers the center of the image but lacks a precise response to specific, particularly small-sized, pest targets, for which the activation signal is weak. In contrast, the MCPF module proposed in this paper generates a feature map with extremely high spatial information fidelity by performing a cascade fusion of multi-scale features from P2 to P5. Its heatmap activation regions are more concentrated and effectively cover the key areas of the target. This result demonstrates that the design of the MCPF effectively overcomes the issue of spatial detail loss in deep networks, providing a higher-quality feature foundation with richer localization information for the subsequent detection neck than that provided by SPPF.

As depicted in Figure 11, a further analysis was conducted on the feature heatmaps output at the same network depth by the standard C2f module and our proposed MSC module. When processing images with complex background textures that are similar to the pest targets, the heatmap of the standard C2f module exhibits large-scale redundant attention on background interference. Its activation area is diffuse and fails to precisely focus on the pest itself. Considering the complex background characteristics of rice paddies, where numerous stems and leaves closely resemble pests in both color and shape, insufficient attention to key target regions may introduce significant interference and increase the risk of false detections. In contrast, the proposed MSC module, through its multi-receptive field parallel branches, can simultaneously capture both the fine-grained details and the broader context of the target. As a result, the activation area in its heatmap is more compact and more accurately focused on the pest target, effectively distinguishing between critical and non-critical regions.

Finally, Figure 12 illustrates the complete process of end-to-end optimization and progressive refinement that the HADF-Net performs on the feature maps. The initial feature map generated by standard convolution (Figure 12B) exhibits a very weak and diffuse activation response, with the entire heatmap showing large blue areas of low response. The process begins in the backbone with the ISAM. After processing (Figure 12C), the activation pattern transforms into a coherent region with a holistic perception of the target, indicating that the model has initially constructed long-range contextual information. Building on this, the enhanced features enter the dense aggregation neck for fusion. At this stage, the feature activation area (Figure 12D) becomes more compact and enriched with detailed information from different levels, which visually demonstrates the effectiveness of HADF-Net’s dense topology in counteracting information dilution. As the culmination of the workflow, this information flow is integrated by the ICAM, where attention is ultimately focused on the most discriminative features that the prediction task relies upon. This forms highly refined activation hotspots with clear contours (Figure 12E). This series of progressive changes clearly showcases how HADF-Net upgrades feature processing from a passive pipeline into an active, multi-level, synergistic, and intelligent perception system that transitions from global enhancement to precise localization.