This study established a rice pest dataset named JRICE-PD. The data collection site was located in Nanchang City, Jiangxi Province (28°40′~29°05′N, 115°45′~116°15′E), one of China’s major rice-growing regions (Yang et al., 2021).

The dataset encompasses 11 common and destructive rice pests, as shown in Figure 1: Curculionidae, Delphacidae, Cicadellidae, Phlaeothripidae, Cecidomyiidae, Hesperiidae, Crambidae, Chloropidae, Ephydridae, Noctuidae, and Thripidae. Field data collection was conducted between June and September 2024 across approximately 50 hectares of paddy fields, covering the rice growing season from tillering to maturation stages. Images were captured using iPhone 13 (12MP, f/1.6 aperture) at shooting distances of 10–50 cm, yielding 2,164 original images at 4032×3024 pixels resolution. Camera settings included auto-focus with exposure compensation of -1.0 to +1.0 EV for varying lighting conditions. Additionally, 2,401 supplementary images were collected from online resources including Google Scholar and Baidu Images, constructing a comprehensive dataset of 4,565 images (Figure 2). All images were resized to 640×640 pixels for training.

Data annotation employed the LabelImg tool following YOLO format with a “tight bounding box” principle (Figure 3). An annotation team comprising two entomological experts and three trained annotators performed annotations under a three-tier quality control mechanism. Annotation consistency was quantified using Intersection over Union (IoU) between independent annotations on 200 randomly sampled images, achieving a mean IoU of 0.91. Annotations with IoU below 0.85 were re-annotated until consensus was reached.

The dataset was divided into training (3,652), validation (456), and test (457) sets at an 8:1:1 ratio.

YOLOv11 represents one of the most advanced single-stage object detection algorithms currently available, incorporating multiple improvements in network structure and feature extraction compared to previous versions. The basic architecture of YOLOv11 comprises three main components: Backbone, Neck, and Head, with its network structure illustrated in Figure 4.

The backbone network is responsible for extracting multi-scale features from input images. It first downsamples the image using initial convolutional layers, then generates feature maps of different resolutions through stacked convolutional layers and specialized modules. The neck network aggregates multi-scale feature maps from the backbone network, fusing and enhancing features before passing them to the detection head. It primarily consists of multiple convolutional layers, C3k2 blocks, Concat operations, and upsampling modules. The neck network first upsamples the low-level features (P5) processed by SPPF and C2PSA to the size of mid-level features (P4) and connects them with P4 features; it then upsamples the connected features to the size of high-level features (P3) and connects them.

The detection head is the final component of the model, responsible for generating prediction results. It receives three features from the neck network, corresponding to high-level, mid-level, and low-level features. The detection head utilizes these three features for focal loss calculation, bounding box detection, and class detection. This design enables YOLOv11 to achieve efficient and accurate object localization and classification in application scenarios.

The BEAM-YOLO network architecture proposed in this study is algorithmically optimized for rice pest detection tasks, establishing an efficient and precise end-to-end detection algorithm through the synergistic action of four innovative modules. The workflow of BEAM-YOLO can be divided into three key stages: feature extraction, feature fusion, and multi-scale detection, with the overall network structure shown in Figure 5.

In the feature extraction stage, the network first establishes initial feature representation through standard convolutional layers, followed by deployment of the MEN module for feature enhancement. The MEN module constructs multi-scale feature representation through the MSF structure while simultaneously reinforcing edge feature extraction, significantly improving the discriminative ability for morphologically similar pests. The Bifurcated Attention Feature Enhancement (BAFE) module integrated at the backbone network terminus utilizes Haar wavelet transform to decompose features into foreground and background components, and establishes contrast enhancement effects through a dual attention mechanism, effectively resolving foreground-background feature confusion problems in complex field environments.

The feature fusion stage employs the EM-BFPN structure to bidirectionally fuse P3, P4, and P5 three-level features extracted by the backbone. EM-BFPN adopts top-down transmission of high-2level semantic information and bottom-up aggregation of refined spatial details. The network integrates the MSCM module for feature processing, which applies convolution kernel combinations of {1,3,5}, {3,5,7}, and {5,7,9} respectively according to the receptive field requirements of different feature levels, forming a gradient multi-scale feature expression. In the feature upsampling process, the innovative SCAU module is employed, significantly expanding the receptive field range without increasing computational burden through depth-separable convolution, channel shuffling, and multi-directional spatial shifting operations.

In the multi-scale detection stage, the network constructs a highly optimized detection head structure, outputting detection results in parallel at P3, P4, and P5 feature levels, achieving precise localization and classification of pest targets of different scales. The three feature levels work collaboratively, forming a detection range covering multiple scales, effectively addressing the detection challenges posed by significant size variations among rice pests.

Conventional convolution operations often neglect fine morphological features of small pests, reducing detection precision. Moreover, edge information progressively attenuates as network depth increases. To address these issues, this study proposes the Morphological Edge Network (MEN) module for restructuring the backbone network architecture, as shown in Figure 6. This module significantly enhances rice pest detection accuracy and environmental robustness through the organic combination of multi-scale feature representation and edge information enhancement mechanisms.

The MEN module is innovatively designed based on the Cross Stage Partial (CSP) structure, organically combining feature extraction with information enhancement mechanisms. This module receives an input feature map X∈ℝH×W×Cin and generates an enhanced feature map Y∈ℝH×W×Cout through complex nonlinear transformations. The overall transformation process can be described by the following unified expression, as shown in Equation 1:

In this formula, X1 and X2 are two parts of features split along the channel dimension after the input feature X undergoes 1×1 convolution, ℳθi represents the MPM transformation function with parameters θi, n denotes the number of enhancement units in the module, and ℱfusion is the final feature fusion function, typically implemented by 1×1 convolution.

Within the module, DPM serves as the basic unit implementing cross-feature enhancement. This unit adopts a dual-path design, significantly enhancing the model’s feature expression capability. Assuming an input feature X, the output feature Z of the unit can be expressed as shown in Equation 2:

where P and Q represent two parallel 1×1 convolution paths, ℰ denotes a tandem sequence composed of MSF units, and G is the fusion function integrating features from both paths. Compared to traditional CSP structures, our design introduces more complex edge-aware mechanisms in the feature extraction path, substantially enhancing the model’s detection capability for pest contours.

MSF focuses on achieving multi-scale feature acquisition and edge information enhancement, with its structure shown in Figure 7. This module adopts a multi-branch parallel architecture, simultaneously processing feature expressions of different abstraction levels. Given an input feature X∈ℝH×W×C, the mathematical expression of the entire module can be unified as shown in Equation 3:

In this expression, ℒ represents the local feature extraction function, implemented through 3×3 convolution; S={s1,s2,…,sk} denotes a set of predefined scale parameters (e.g., {3,6,9,12}); As is the adaptive pooling and feature transformation function at scale s; Ts represents the transformation function including upsampling and edge enhancement; and Φ is the final feature fusion function.

In the multi-scale feature path, the EdgeEnhancer submodule plays a crucial role, designed to strengthen target edge information and enhance the model’s perception of pest contours. Given an input feature X, the processing of EdgeEnhancer can be mathematically formulated as shown in Equation 4:

In this formula, Pavg represents the average pooling operation, used to simulate background information of local regions; X−Pavg(X) calculates the difference between the original feature and its smoothed version, explicitly extracting edge information; ℋ is a nonlinear transformation function composed of convolutional layers; σ denotes the Sigmoid activation function, mapping edge features to the [0,1] interval as attention weights; and ⊙ represents the Hadamard product (element-wise multiplication). Through this adaptive attention mechanism, the module can precisely enhance feature responses in pest edge regions while suppressing background noise, significantly improving detection accuracy and robustness.

The MEN module proposed in this study, by parallel integration of adaptive average pooling operations with different receptive fields, constructs a feature extraction path with hierarchical multi-scale representation capabilities, effectively capturing discriminative feature information of pests of different sizes. The EdgeEnhancer design in this module, through efficient extraction and enhancement of edge information, significantly improves the model’s discrimination capability for fine morphological features of pests, demonstrating superior recognition performance and classification accuracy especially for morphologically similar pest species.

Although the Cross-Stage Partial Spatial Attention (C2PSA) module has achieved remarkable success in object detection, it struggles to discriminate features between pest targets and rice plants, resulting in false positives and false negatives in complex field environments. Single spatial attention mechanisms struggle to capture multi-scale morphological variation features of pests. To address these issues, this paper proposes a novel Bifurcated Attention Feature Enhancement (BAFE), with its structure shown in Figure 8. This module significantly improves rice pest detection precision through wavelet transform separation of foreground and background information and the introduction of a dual attention mechanism.

The BAFE module employs frequency domain deconstruction and contrast-driven cascaded attention mechanisms, achieving efficient feature enhancement and aggregation. Given an input feature map X∈ℝB×C×H×W, the module first performs initial feature optimization through an input preprocessing network Φin, then obtains enhanced features Y through a series of transformation operations. The core component HaarWaveletConv (Su et al., 2024) achieves frequency domain decomposition of features based on discrete Haar wavelet transform principles. This component maps input features to different subbands in the wavelet domain through a set of predefined convolution kernels. Specifically, for input feature X, the wavelet transform process can be expressed as shown in Equation 5:

where Ψa, Ψh Ψv, and Ψd represent Haar filters for approximation (low-pass), horizontal, vertical, and diagonal (high-pass) directions respectively, and ∗ denotes the convolution operation.

Specifically, the HaarWaveletConv employs fixed (non-learnable) Haar wavelet filters initialized according to the standard discrete Haar transform coefficients: Wa=12[1,1;1,1], Wh=12[−1,−1;1,1], Wv=12[−1,1;−1,1], and Wd=12[1,−1;−1,1]. The convolution operation uses a stride of 2 and no padding, effectively downsampling the feature map by a factor of 2 while decomposing it into frequency subbands. These filter weights remain frozen during training to preserve the mathematical properties of the Haar wavelet transform.

Among these four subbands, the approximation subband Wa contains low-frequency structural information of the image, while the remaining three high-frequency subbands {Wh,Wv,Wd} capture edge and texture details in different directions. In our implementation, these high-frequency subbands are fused into a single high-frequency feature representation Fhigh=Wh+Wv+Wd, which, together with the low-frequency feature Flow=Wa, provides the foundation for subsequent contrast-driven processing. After obtaining frequency domain decomposed features, the module uses a dual attention mechanism to process high-frequency and low-frequency information separately. The foreground attention stage first constructs a spatially sensitive attention map based on high-frequency features. Given the deformed feature V∈ℝB×Nh×L×k2×dh foreground attention calculation can be represented as shown in Equation 6:

where F˜high is the high-frequency feature after pooling downsampling, Qfg and Kfg are query and key transformation functions respectively, B is positional encoding, Nh is the number of attention heads, L is the number of feature spatial positions, k is the convolution kernel size, and dh is the feature dimension of each head. This attention mechanism enhances perception of edges and textures of small pests by considering local spatial dependencies. After applying attention weights to value feature V, an enhanced feature representation is obtained as shown in Equation 7:

where ⊙ represents broadcast element-wise multiplication, and ℱproj is projection transformation. Similarly, the background attention stage utilizes low-frequency features to guide further enhancement of the first stage output, forming the final feature representation. This process can be formalized as shown in Equations 8, 9:

This cascaded contrast attention design enables the module to progressively refine feature representations under the guidance of different frequency domain information, particularly suitable for capturing subtle differences of rice pests in complex backgrounds. Notably, by using sliding window (unfold) operations and position-sensitive attention mapping, the module can efficiently process spatial dependencies, enhancing representation capability for small targets and pests with complex morphologies.

The BAFE module proposed in this paper achieves adaptive decomposition of features through wavelet transform, effectively separating low-frequency background information from high-frequency foreground information, resolving the foreground-background feature confusion problem in traditional detection networks and enabling the model to focus more precisely on pest targets. It also designed a dual attention mechanism, processing foreground and background information separately and forming a contrast enhancement effect through a stacked approach, significantly enhancing the model’s recognition capacity for morphologically variable and minuscule rice pests.

Traditional neck networks exhibit evident limitations in feature fusion, typically employing simple concatenation or weighted summation without adequately considering inter-layer correlation and complementarity. This results in poor detection performance for small, dense, and morphologically variable targets such as rice pests. To address these issues, this paper proposes an Enhanced Multi-scale Bidirectional Feature Pyramid Network (EM-BFPN), with its structure shown in Figure 9. This network achieves efficient information interaction and complementary information mining between different feature layers through the design of an Adaptive Feature Fusion Mechanism (AFFM) and Multi-scale Convolution Module (MSCM).

The workflow of EM-BFPN can be divided into three stages: feature preprocessing, multi-scale bidirectional feature fusion, and feature enhancement. In the feature preprocessing stage, P3, P4, and P5 feature maps from the backbone network are first unified in channels through 1×1 convolution to reduce computational complexity and improve feature fusion efficiency. In the multi-scale bidirectional feature fusion stage, EM-BFPN implements bidirectional feature transmission from top-down and bottom-up, forming a closed-loop feedback mechanism. The feature fusion process is implemented by the AFFM mechanism, which performs weighted fusion of features from different sources through learnable weight coefficients. The feature enhancement stage employs the innovative MSCM module, which can extract features under multiple receptive fields, effectively enhancing the model’s adaptability to pests of different scales. Finally, P3, P4, and P5 features enhanced through multi-level feature interaction are sent to the detection head for target classification and localization.

The AFFM module in EM-BFPN is a key component for achieving efficient feature fusion. Unlike simple feature concatenation or summation, this module adaptively adjusts the contribution of different features through learnable weight parameters.

For a set of input features {F1,F2,…,Fn}, a learnable weight parameter wi is first defined for each input feature. Then, {w1',w2',…,wn'} are obtained through ReLU activation, ensuring that each input feature’s contribution in the fusion process is non-negative, avoiding mutual cancellation between features. Finally, after weight normalization, the weighted sum of input features is output as shown in Equation 10:

where ϵ=10−4 is a small constant to prevent division by zero. Compared to fixed-weight fusion methods, this learning-based fusion mechanism can dynamically adjust the importance of different features, adapting to different detection scenarios. Through end-to-end training, weight parameters can be automatically optimized according to the loss function without manual adjustment, maintaining computational efficiency.

The MSCM module integrates the advantages of cross-stage partial networks and multi-scale convolution, effectively extracting and processing multi-scale feature information, with its structure shown in Figure 10. Given an input feature X∈ℝC1×H×W, the forward propagation process of MSCM can be represented as shown in Equation 11:

where ϕ1,ϕ3:ℝC1→ℝC1·e/2 and ϕ2:ℝC1·e/2→ℝC1·e/2 represent feature transformation functions respectively, e is the expansion coefficient, Concat[·] denotes feature concatenation operation along the channel dimension, and ∏i=1nDSARBi represents the combined operation of multiple cascaded Dynamic Scale-Adaptive Residual Blocks (DSARB).

DSARB employs parallel multi-scale depth-separable convolution to enhance feature extraction capability. The complete computation process of DSARB can be represented as shown in Equation 12:

where ℱexp:ℝCin×H×W→ℝe·Cin×H×W is pointwise convolution for channel expansion, e is the expansion coefficient; _ℳ is the multi-scale feature aggregation function, representing the feature aggregation result after the Parallel Receptive Field Fusion Module (PRFFM); Ψ represents the Channel Shuffle operation (Zhang and Yang, 2021); ℱproj:ℝe·Cin×H×W→ℝCout×H×W is pointwise convolution for channel projection; ℱres:ℝCin×H×W→ℝCout×H×W is 1×1 convolution for residual connection; and s represents the convolution stride.

The PRFFM module is the core component of DSARB, implementing parallel multi-scale depth-separable convolution. Given an input feature X∈ℝC×H×W and a set of convolution kernel sizes K={k1,k2,…,km}, the output of PRFFM can be represented as shown in Equation 13:

where ℱDWki:ℝC×H×W→ℝC×H'×W' represents the depth-separable convolution operation with kernel size ki: where Wcki∈ℝ1×ki×ki is the convolution kernel weight of the c-th channel, * represents the two-dimensional convolution operation, ℬN represents batch normalization, and σ represents the ReLU nonlinear activation function.

The EM-BFPN neck network structure proposed in this paper has been specially optimized for rice pest detection scenarios. Through multi-path feature flow and iterative feature fusion, it significantly enhances feature reuse efficiency and multi-scale information interaction, addressing the limitations of traditional FPN in processing rice pest targets with significant scale variations. By adopting the AFFM fusion mechanism, the network can adaptively adjust the importance of features at different scales, enhancing selective extraction of key information and avoiding information redundancy and noise interference problems caused by traditional simple fusion methods.

Traditional upsampling modules typically employ bilinear or nearest-neighbor interpolation. While computationally efficient, these methods often cause information loss and spatial blurring during feature map reconstruction. Particularly in rice pest detection tasks, where pest targets are typically small in volume, morphologically similar, and in complex backgrounds, such information loss significantly affects detection precision. To address these issues, this paper proposes an upsampling module called Spatial-Channel Augmented Upsampling (SCAU), as shown in Figure 11. This module effectively enhances the model’s capability to detect small-sized rice pests and distinguish between similar pest categories by employing a Channel Shuffle mechanism and innovative Multi-Directional Feature Shifting (MDFS) units.

The workflow of the SCAU module primarily includes four stages: upsampling, Channel Shuffle, Multi-Directional Feature Shifting, and pointwise convolution. The input feature map X is processed through an upsampling layer combined with depth-separable convolution (DWConv), then processed through the Channel Shuffle mechanism to obtain X". This operation ensures that channel information from different groups can be thoroughly mixed, enhancing the diversity of feature expression. After channel shuffling, the feature map enters the MDFS unit for spatial shift mixing, followed by further fusion of channel information through pointwise convolution (PWConv) to obtain the final output map.

The MDFS unit is the core innovation of the SCAU module, aimed at enhancing the spatial context perception capability of feature maps without increasing parameter count. For an input feature map X∈ℝC×H×W, the MDFS unit first evenly divides it along the channel dimension into four sub-feature maps, obtaining X1,X2,X3,X4. Each sub-feature map Xi∈ℝC4×H×W, i∈{1,2,3,4}, contains one-quarter of the channels of the input feature map.

After division, different directions and magnitudes of circular shift operations are applied to these four sub-feature maps, as shown in Equations 14–17:

In these formulas, Roll represents the circular shift operation, TH,s and TW,s represent shift transformation functions in height and width dimensions respectively, and s is the shift amount. Specifically, dims=2 indicates shifting in the height dimension, and dims=3 indicates shifting in the width dimension. Positive shift values indicate downward or rightward movement of features, while negative shift values indicate upward or leftward movement. After the shift operations, the four sub-feature maps are reconnected along the channel dimension, as shown in Equation 18:

where Concat represents the tensor concatenation operation, and dim=1 specifies concatenation along the channel dimension. The final output feature map Xout has the same shape as the input feature map X but contains enhanced spatial context information.

The SCAU module proposed in this paper addresses small target recognition and similar category discrimination problems in rice pest detection tasks through upsampling combined with depth-separable convolution, channel shuffling mechanism, and spatial shift mixing strategy, effectively enhancing feature map expression capability without significantly increasing computational complexity. Compared to traditional upsampling methods, the SCAU module reduces computation while preserving spatial information by employing depth-separable convolution. The module adopts a channel shuffling mechanism to promote interactive fusion of different channel features, enhancing the model’s representation capability.

The hardware configuration for this experiment comprised computing nodes and graphics processors, with specific environmental settings and experimental parameters detailed in Tables 1, 2. The central processing unit utilized was a 12th Gen Intel(R) Core(TM)i7-12650H, while the graphics processor employed was an NVIDIA GeForce RTX 4060. The software environment was deployed on the Windows 11 operating system, with a programming environment based on Python 3.9 and the PyTorch 11.7 deep learning framework. Specific parameters for model training were configured as follows: iteration period (epoch) was set to 250, batch size to 16, and the optimization algorithm utilized was Stochastic Gradient Descent (SGD) with an initial learning rate of 0.01 and a momentum factor of 0.937, with other parameters adopting default values. Input data dimensions were uniformly adjusted to 640×640 pixel resolution through standardized preprocessing.

To verify the effectiveness of the proposed BEAM-YOLO algorithm in rice pest detection tasks, we conducted ablation experiments on four core innovations. Here, A represents the MEN module, B represents the BAFE module, C represents the EM-BFPN module, and D represents the SCAU module. Starting from the baseline model, each innovative module was progressively added to evaluate their independent and combined contributions to detection performance. The ablation experiment results are shown in Table 3.

The experimental results clearly demonstrate the significant performance improvement brought by the proposed innovative modules. When all four modules were combined, our proposed BEAM-YOLO model achieved 86.6% mAP@50, a 3.3% improvement over the baseline, and reached 72.7% under the more stringent mAP@50–95 evaluation criterion, a 3.0% improvement. Notably, the combination of the MEN module and EM-BFPN achieved significant performance improvement while maintaining relatively low computational complexity and parameter count. The BAFE module showed exceptional performance in improving Recall, enabling the model to better detect difficult-to-recognize pest targets. Figure 12 illustrates the ablation experiment results for different network component combinations through a three-dimensional bar chart, intuitively presenting the performance of each configuration on two key metrics: mAP@50 and mAP@50-95. A clear progressive improvement trend in model performance can be observed as components are accumulated.

To comprehensively evaluate the practical performance of the BEAM-YOLO model in rice pest detection tasks, this study comparatively analyzed the detection results of this model and mainstream YOLO series models in multiple typical scenarios, as shown in Figure 15. The experimental design included four representative detection scenarios: (1) Ephydridae pests in green leaf backgrounds, for evaluating model precision in target recognition under similar backgrounds; (2) tiny Hesperiidae pests on green leaves, for testing the model’s ability to capture fine morphological features; (3) extremely small-sized Thripidae pests, for examining the sensitivity of detection models to minuscule targets; and (4) multiple Noctuidae pests in complex backgrounds, for verifying model robustness in multi-target complex environments. Through these challenging representative scenarios, the algorithm evaluated the performance differences between the BEAM-YOLO constructed based on four innovative modules and existing YOLO variants.

The experimental results intuitively demonstrate the detection advantages of the BEAM-YOLO model in various complex scenarios. In scenario (1), BEAM-YOLO’s detection confidence for Ephydridae pests reached 0.95, higher than other models, particularly 0.21 higher than YOLOv3-tiny; in scenario (2) with extremely small Hesperiidae targets, BEAM-YOLO’s recognition confidence reached 0.95, while YOLOv6 incorrectly identified it as Cecidomyiidae, and YOLOv5 exhibited confusion with double tagging; in scenario (3), all models performed relatively similarly, but BEAM-YOLO still led with the highest confidence of 0.96; in the most challenging scenario (4), BEAM-YOLO successfully detected all Noctuidae targets with more balanced confidence distribution, while YOLOv6 and YOLOv3-tiny misclassified some targets as Ephydridae, and YOLOv10, YOLOv9, and YOLOv5 showed obvious missed detections. These results fully validate the synergistic effect of the innovative modules in BEAM-YOLO: the MEN edge enhancement module precisely captures pest morphological features, the BAFE contrastive wavelet attention effectively separates pest targets from complex backgrounds, and the EM-BFPN feature pyramid network and SCAU upsampling module significantly enhance multi-scale feature extraction and fusion capabilities, giving BEAM-YOLO comprehensive performance advantages in detection tasks involving small targets, similar morphologies, and complex backgrounds like rice pests.