The rice samples used in this study were collected from the rice germplasm resources team at the Institute of Crop Sciences, Chinese Academy of Agricultural Sciences. Field data collection was conducted in September 2024 at the Yichun Gao’an Base in Nanchang, Jiangxi Province (28.3°N, 115.1°E), where phenotypic data were acquired from 1,772 rice germplasm accessions at maturity, including 1,338 japonica and 434 indica varieties, with a total of 4,700 panicle images collected. Supplementary validation data were collected from the Modern Agricultural Science and Technology Innovation Demonstration Park of Sichuan Academy of Agricultural Sciences (30.8°N, 104.2°E), an independent cross-region validation base—covering 200 local rice germplasm accessions (120 japonica and 80 indica varieties) with a total of 600 panicle images (exclusively for generalization verification, not involved in model training). Post-harvest panicle imaging was performed indoors under LED lighting (Figure 1). A smartphone mounted at a fixed distance of 30 cm from the samples captured images at a resolution of 3024×4032 pixels, with automatic ISO and shutter speed adjustment. Panicles were placed on a uniform black background alongside a 30 mm-diameter red calibration board to enable pixel-to-physical size conversion. A white numbered plate corresponding to field plot IDs ensured data consistency with planting records.

All captured images underwent a manual preliminary screening conducted by two trained researchers to ensure data quality. The screening criteria were to discard any image exhibiting: (1) significant motion blur where grain edges were not clearly visible; (2) poor focus across the majority of the panicle; (3) severe overexposure or underexposure that obscured grain details. This qualitative but standardized filtering step was crucial for removing unsuitable samples before further processing. Remaining images were cropped to isolate panicle regions and renumbered, yielding 4,700 high-quality images for subsequent processing.

When using deep learning to obtain rice panicle traits, manually measured data of rice panicle traits are required to provide accurate real - world data for model training and testing. Therefore, when manually measuring rice panicle traits, we tried our best to keep the panicles intact, avoiding breaking or damaging them, and placed the panicles flat on the table. (1) Regarding the measurement of panicle length, 400 rice panicle samples were selected. A ruler with an accuracy of 1 mm was used to measure the length from the base to the top of the panicle parallel to the panicle to obtain the panicle length data. (2) For the counting of grains on the panicle, the number of grains on 600 rice panicles was manually counted. (3) When measuring the length and width of panicle grains, 240 rice panicle samples were selected, and a vernier caliper was used to measure the grains on the panicles. (4) In terms of evaluating the maturity of rice panicles, three rice breeding experts manually evaluated the maturity of rice panicle images to determine the maturity level of the panicles. (5) For the identification of rice panicle types, three rice breeding experts manually classified 3000 selected rice panicle images, with the classification criteria and typical panicle morphologies shown in Figure 2. The first type was “loose”, where the grains had slight mutual occlusion; the second type had the characteristic of “normal”, with a large amount of mutual occlusion between grains; the third type was “dense”, with severe mutual occlusion between grains, and the appearance was similar to a rod.

In this study, the 4,700 rice panicle images were partitioned according to their specific applications. A subset of images was annotated using LabelImg for distinct analytical purposes, with detailed allocations presented in Table 1. (1) A total of 1,000 images were designated for grain detection and performance evaluation across different models, specifically allocated as 700 for training the grain detection model, 200 for validation, and 100 for testing. (2) Among these, 3,000 images were utilized for panicle category detection, comprising 2,100 for training, 600 for validation, and 300 for testing. (3) From the 2,100 manually measured sample images, 400 were employed to validate the panicle length extraction algorithm, 660 for regression modeling, 240 for verifying the length-width extraction algorithm, and 1,200 for panicle grain maturity analysis.

Accurate detection of individual rice grains is the foundation for multiple trait extraction tasks, including grain counting, dimension measurement, and maturity analysis. To achieve this with high accuracy and speed, we proposed an improved OPG-YOLOv8 detection algorithm, with its architecture detailed in Figure 4E. This algorithm enhances the standard YOLOv8 model by primarily optimizing its neck and head structures (Jocher et al., 2023).

Specifically, we introduced the CBAM (Convolutional Block Attention Module) to suppress background noise in shallow-layer features (Woo et al., 2018). The CBAM is an attention mechanism composed of a channel attention module (CAM) and a spatial attention module (SAM). By selectively weighting the channel and spatial information, it significantly enhances the feature representation ability for small targets like rice grains. Furthermore, a higher-resolution feature map is introduced in the neck for multi-scale feature fusion to compensate for information loss during feature transmission. Concurrently, an additional detection head is added to the head of OPG-YOLOv8, which utilizes the high-resolution feature map for prediction, effectively improving the detection accuracy of small targets. This optimized model serves as the core engine for all subsequent grain-level phenotypic analyses.

Under natural conditions, the mutual occlusion of grains within a panicle poses a significant challenge to accurate counting based solely on object detection. Denser panicles can have a substantial number of grains hidden from view, leading to systematic underestimation. To address this challenge, we developed a two-stage method that integrates density classification with regression modeling to correct the initial count.

The first stage involves classifying the overall panicle morphology to estimate the potential level of grain occlusion. Based on expert evaluation, we defined three density categories reflecting the degree of mutual grain occlusion: “loose”(slight occlusion),”normal”(moderate occlusion), and” dense”(severe occlusion), with typical examples shown in Figure 2. The OPG-YOLOv8 model was trained on a dedicated dataset to automatically recognize the visual patterns of these types and assign an input panicle image to the most appropriate category. This classification provides the crucial contextual information needed for the subsequent correction step.

The OPG-YOLOv8 model was trained on a dedicated dataset to automatically recognize and assign an input panicle image to one of these three categories.

In the second stage, the classification result from Stage 1 is used to apply a targeted correction to an initial grain count. First, the OPG-YOLOv8 object detection model is used to obtain a preliminary predicted count of all visible grains in the panicle (denoted as Xi). Then, based on the panicle’s assigned density category, the corresponding pre-established linear regression equation is selected to calculate the final, corrected grain count (denoted as Yi).

This regression model was built to empirically correct for the systematic underestimation of grain counts caused by occlusion in denser panicles (Equations 1–3). The derivation and validation process was as follows: a dedicated regression modeling sub-dataset of 660 panicle images (220 for each density category) with manually verified, ground-truth grain counts was utilized. First, the OPG-YOLOv8 object detection model was applied to this dataset to obtain the initial predicted count of visible grains (denoted as X_i) for each image.

Subsequently, for each density category, a simple linear regression analysis was performed by fitting these predicted counts (X_i) against their corresponding manual ground-truth counts using the least-squares method. The resulting equations represent the best-fit linear models that map the visible grain count to the estimated total grain count for each respective category. The equations for the three density classes (1-loose, 2-normal, 3-dense) are presented as follows:

In this framework, X_i represents the initial predicted grain count from the object detection model for a panicle in category i, while Y_i represents the final, more accurate corrected grain count. The validation of these models is demonstrated by the high coefficient of determination (R²) values presented in the Results section (Figure 5), which confirm a strong goodness of fit and a significant linear relationship between the predicted and true values. This two-stage approach allows the system to systematically compensate for hidden grains based on the panicle’s structure, significantly improving the overall counting accuracy.

To accurately measure the panicle length, this study proposed a panicle length extraction algorithm based on the dynamic pruning strategy of the depth-first search (DFS) algorithm. The calculation process of this method is shown in Figure 4F. First, the image is grayscaled, and then it is filtered to remove noise, eliminating interferences caused by the variability of lighting conditions and the physical limitations of the camera sensor. Subsequently, threshold segmentation is performed to separate the panicle part from the background. Finally, morphological closing operations are carried out to fill holes and connect edges, prominently highlighting the main characteristics of the panicle.

The skeletonization function in the skimage library (Wu et al., 2019) is used to extract the central axis of the panicle. To adhere to standard agronomic measurement practices, the panicle length was defined as the distance from the panicle neck to the furthest tip of the panicle. In our algorithm, the panicle neck is automatically identified as the lowest branching point on the extracted skeleton. This ensures that the main panicle stalk below this point is consistently excluded from the length calculation.

Following skeletonization, the DFS pruning algorithm (Song et al., 2019) is employed for path planning to depict the main path of the panicle. The dynamic pruning strategy of the DFS algorithm constructs a two-dimensional matrix from the pre-processed panicle image. One end of the skeleton is selected as the starting point, and the DFS algorithm is used to traverse the graph structure of the panicle (Yao et al., 2017). During the search process, the length of the current path is recorded. When a new node is searched, the length of the current path is compared with the recorded optimal length. If the length of the current path is less than the optimal length, the search continues; if it is greater than or equal to the optimal length, pruning is performed, and the search for this path is no longer continued. According to the recorded path information, the main path of the panicle can be depicted on the original image or a new image, and then the actual panicle length is calculated based on the pixel-to-physical size conversion factor (Hu et al., 2016).

Based on the position information of the grains in the panicle detected by the OPG - YOLOv8 model proposed in this study, the bounding box information of the rice panicle grains is determined according to the detection results, and then the length and width are extracted. However, due to the occlusion of grains in the panicle, it may happen that the detection box does not completely contain the rice panicle grains. Therefore, the detection results need to be screened. When screening the grains, Conf (confidence threshold) is used to measure the confidence level of the model’s prediction of the detected target. Its value range is usually between 0 and 1. The higher the value, the more confident the model is about the accuracy of the detection result. Conf is used to determine whether the prediction box accurately covers the target grain. Only the boxes that exceed the set threshold are considered valid samples. After repeated screening and verification in this experiment, the rice panicle grains with a confidence threshold greater than 0.8 are further measured for length and width. After segmenting the selected grains, the length and width of the grains are extracted. The extraction process is shown in Figure 4G. It is important to note that the initial object detection model provides a bounding box primarily for locating each grain, which is not used for direct measurement. This is because initial bounding boxes can sometimes fail to capture the entire grain, particularly slender tips or awns, due to occlusion or morphological variations, which could introduce measurement inaccuracies.

To ensure high-precision measurement, our method employs a more robust two-step process. First, the coordinates of the initial bounding box are used to isolate a region of interest (ROI) containing the target grain. Second, within this ROI, we perform a precise threshold segmentation and particle filtering to extract the exact pixel morphology of the grain body. Finally, the minimum bounding rectangle is calculated based on this accurate segmentation mask, not the initial detection box. This segmentation-based approach ensures that the final length and width measurements accurately reflect the true dimensions of the grain by systematically overcoming potential inaccuracies in the initial detection phase.

To objectively quantify grain maturity, a method based on the color characteristics of the panicle was developed. While the RGB color space from digital images captures fundamental color information, it is not perceptually uniform, meaning the numerical distance between two RGB values does not directly correspond to the human-perceived color difference. To establish a more robust and human-centric metric, a multi-step workflow was designed to convert the initial RGB data into the Hunter Lab color space, which is approximately uniform for human color perception. This ensures that the final calculated yellowness index (YI) provides a consistent and meaningful measure of maturity.

The yellowness extraction for grains leverages the detection results from the OPG-YOLOv8 model. For each detected grain, the calculation process begins with its average RGB values normalized via Equation 4, and proceeds as follows:

Since the human eye’s perception of color is nonlinear, and the RGB color space is a linear space based on physical devices, in order to more accurately simulate the visual characteristics of the human eye, gamma correction is required and implemented via Equation 5. The specific formula is as follows:

(the same approach applies to g and b)

After gamma correction, the normalized and corrected RGB values are converted to CIE XY values using the following formula (Equation 6):

In the CIE XYZ color space, X, Y, and Z represent distinct color components. Based on the obtained CIE XYZ values, the Hunter Lab parameters are calculated. In the Hunter Lab color space, L denotes Lightness, which reflects the brightness of the color calculated by Equation 7; a represents Chromaticity in the red-green axis, indicating the color’s shift along the red-green direction; and b signifies Chromaticity in the yellow-blue axis, representing the color’s shift along the yellow-blue direction. The formulas for calculating these parameters are as follows:

These transformations from the RGB color space to the Hunter Lab space follow the standard colorimetric principles and formulas established by the International Commission on Illumination (CIE) (CIE, 2004). Finally, the yellowness value is calculated based on the aforementioned data results, using the following formula:

Through this formula Equation 8, the information of lightness (L) and chromaticity (a, b) is integrated, enabling the precise quantitative calculation of yellowness. This yields a numerical value that accurately reflects the degree of yellow in the color.

Based on the comparison between the rice maturity results judged by rice breeding experts and the average yellowness values, this study established a clear standard for the classification of rice maturity stages, as presented in Table 2: