The experiment was conducted during 2022–2023 at the Chongzhou Experimental Base of Sichuan Agricultural University (103°39’E, 30°33’N), as depicted in Figure 1A . This region has a subtropical monsoon climate, with an average temperature of 16.2°C, an annual total sunshine duration of 1400 hours, and an annual total rainfall of 918 mm. The basic chemical properties of the 0–20 cm soil layer at the experimental site were as follows: organic matter content, 24.3 g·kg^-1; total potassium, 15.2 g·kg^-1; total nitrogen, 1.6 g·kg^-1; total phosphorus, 1.3 g·kg^-1; available potassium, 169.4 mg·kg^-1; available nitrogen, 299.5 mg·kg^-1; and available phosphorus, 36.5 mg·kg^-1.

The experimental materials consisted of 240 soybean varieties (a natural population formed by four provinces in China: Sichuan, Yunnan, Chongqing, and Guizhou) ( Appendix 3 ). As shown in Supplementary Figure S1 , the genetic diversity of this population is extremely extensive, and all of these populations were planted in 2022. In 2023, based on the previous year’s yield, extremely low-performing varieties were excluded, resulting in the final cultivation of 202 soybean varieties. Over the two years of field experiments, each variety was subjected to three replications and two planting methods to increase yield differences (sole cropping and strip intercropping with corn). In the strip intercropping system with corn, the corn variety used was Zhongyu 3, a semicompact spring corn. The planting materials used were provided by the Crop Strip Composite Planting Engineering Technology Research Center of the College of Agronomy, Sichuan Agricultural University. The field layout is shown in Figures 1B, C . In the corn–soybean strip intercropping system, two rows of corn (corn strip) were intercropped with two rows of soybean (soybean strip), with a length of 20 m and a width of 2 m (the row spacing for both the corn-corn and soybean-soybean systems was 40 cm, and the spacing between the corn and soybean strips was 60 cm). Both corn and soybean were single-hole planted, with a hole spacing of 20 cm for corn. For soybean, pots were planted with a diameter of 25 cm, a diameter of 20 cm, and a height of 25 cm; the pots were filled with 10 kg of soil. Under the intercropping treatment, the potted soybeans were placed in wide rows of corn, with two pots placed side by side in each strip. The soybean plant density and row spacing under the monocropping treatment were consistent with those under the intercropping treatment. The base fertilizer for corn was compound fertilizer (N:P:K = 13:5:7) applied at a rate of 923 kg·hm^-2. At the jointing and tasseling stages, urea (N ≥ 46%) was applied at rates of 98 kg·hm^-2 and 163 kg·hm^-2, respectively. No fertilizer was applied throughout the entire growth period of the soybeans.

The flowchart depicted in Figure 2 illustrates the methodology employed in this study for obtaining high-throughput soybean phenotypic data. This process included image acquisition, image segmentation (offline model training and online image segmentation), and parameter extraction. The details of this flowchart are discussed below. In this study, a total of three categories of parameters were extracted from two shooting angles, including 6 color parameters each for the top and side views, 17 textural parameters each for the top and side views, and 16 and 30 morphological parameters for the top and side views, respectively (refer to Appendix 1 ). In this research, the color, texture, and morphological parameters from the top and side views were individually or collectively used to estimate soybean yield.

After the soybeans matured, manual collection and recording of individual soybean yields were conducted. The yield distribution is depicted in Figure 3 , and the data for both years follow a normal distribution.

We evaluated the broad-sense heritability (H² ) of the data over the two years using the formula H² = V_g/(V_g + V_ll/n + V_ly/n + V_r/n²). Here, V_g represents the genotypic variance, V_ll represents the variance between varieties and locations, V_ly represents the variance between varieties and years, and V_r represents the residual variance, with n denoting the number of replicates. Means and standard deviations were calculated using BLUP values, and the statistical significance of P values was determined using paired t tests.

To further investigate the estimation accuracy of the yield prediction models, data preprocessing was performed using R Version 4.1.1 (R Foundation, Vienna, Austria). The outlier test function from the car package was utilized to remove outliers. A total of 149 outliers were eliminated from the 1440 datasets for 2022, and 121 outliers were removed from the 1212 datasets for 2023. Subsequent analyses were based on the remaining 2382 datasets. The statistical modeling was conducted using Python 3.7. In this study, five different regression methods were employed: CatBoost, LightGBM, RF, GBDT, and MLP. All the input parameters were normalized. The two years of data were divided into training and testing datasets based on the years. The samples from 2022, totaling 1291, were used as the training set, while those from 2023, totaling 1091, were used as the testing set. Tenfold cross-validation was also performed. All the numerical data were plotted using Origin 2019 and SPSS 2018. The data analysis methods employed in this study are illustrated in Figure 4 .

The red area indicates a negative correlation, while the blue area indicates a positive correlation. Lighter colors represent weaker correlations. As shown in Figure 5 , there were a total of 23 image indicators in the sideview that were absolute correlations (hereinafter referred to as correlations) with a yield greater than 0.5. Among the sideview color parameters, all the indicators were correlated with a yield less than 0.5 ( Figure 5A ). Among the sideview morphological parameters, 13 indicators had a correlation with a yield greater than 0.5, and except for SPA2 and SCA, the other 11 indicators were positively correlated with yield ( Figure 5B ), indicating that an increase in these 11 indicators leads to an increase in soybean yield. Among the sideview textural parameters, 10 indicators were correlated with a yield greater than 0.5. Among them, SGD, SIG1, SE, and SDM were negatively correlated with yield, while SG, SGA, SGE1, SGE2, SME, and SHE were positively correlated with yield ( Figure 5C ).

In the top view, a total of 16 image indicators were correlated with a yield greater than 0.5. According to the top-view color parameters, all the indicators were correlated with a yield less than 0.5 ( Figure 5D ). Among the top-view morphological parameters, 9 were correlated with a yield greater than 0.5, and all the indicators were positively correlated with yield ( Figure 5E ). Among the top-view textural parameters, 7 indicators exhibited a correlation with a yield greater than 0.5. Among them, TE, TDM, and TGH had negative correlations with yield, while TG, TGE1, TME, and THE had positive correlations with yield ( Figure 5F ).

An interesting phenomenon was observed here: the relationship between the image parameters and yield was quite similar between the side and top views. For example, SE and SDM exhibited a high negative correlation with yield, corresponding to TE and TDM, which also exhibited a high negative correlation with yield in the top-view image parameters. Similarly, SG, SGE1, SME, and SHE exhibited a high positive correlation with yield, corresponding to TG, TGE1, the TME, and the, which also showed a high positive correlation with yield in the top-view image parameters. Therefore, further exploration of the relationships between side- and top-view image parameters and soybean yield is particularly intriguing.

The sideview image parameters with Pearson coefficients greater than 0.5 were selected for soybean yield estimation. In this study, the CatBoost, LightGBM, GBDT, RF, and MLP models were used to estimate soybean yield, and three combinations were trained and evaluated. The three combinations of input variables were named m1-m3, where m1 consists of morphological parameters (SPA1, SCA1, 1/5MW, 2/5MW, 3/5MW, 4/5MW, SMW, TC2, SMIW, SRA, SSC, SPA2, and SCA), m2 consists of textural parameters (SMG, SGD, SIG1, SE, SG, SGA, SGE1, SGE2, SME, and SDM), and m3 consists of morphological and textural parameters.

As shown in Table 2 , the R² value on the training set was generally greater than that on the test set, and the RMSE and MAE were generally lower than those on the test set. For m1, the prediction accuracy of the five models on the test set fluctuated greatly, with 0.42 ≤ R² ≤ 0.77, and the GBDT and LightGBM models achieved the best prediction accuracy, both with R² values of 0.77. The performance accuracy of RF was the lowest, and the prediction accuracies of CatBoost and MLP were the lowest (R² ≤ 0.52), with the highest errors. For m2, the overall prediction accuracy of the five models was lower than that for m1, but the GBDT model still achieved the highest prediction accuracy, with R^{2 =} 0.73, while CatBoost and MLP yielded the lowest prediction accuracies, with R² values of 0.49 and 0.44, respectively. For m3, after combining textural parameters with morphological parameters, the overall prediction accuracy of the five models slightly improved, with the MLP model showing the most significant improvement. According to Figures 6A–I , the error order is m2 > m1 > m3, and the R² order is m3 > m1 > m2. With the enrichment of indicators, the prediction accuracy increased, and the error decreased, with morphological parameters leading to better estimation performance than textural parameters. Overall, in the prediction of soybean yield based on sideview image parameters, the optimal predictive model was GBDT.

Figure 6 shows the statistical results of cross-validation for soybean yield prediction based on side-image parameters. It can be seen that m3 still performed the best overall, with the highest prediction accuracy for the GBDT model, followed by LightBoost. However, compared to the prediction results in Table 2 , the overall prediction accuracy improved.

The soybean yield was estimated using top-image parameters with a correlation greater than 0.5. The three combinations of input variables were named M1-M3; M1 consisted of morphological parameters (TPA1, TCA1, TRW, TRH, TCR, TRA, TCA2, TSC, and TCA3), M2 consisted of textural parameters (TGD, TE, TG, TGE1, TME, TDM, and TGH), and M3 consisted of a combination of morphological and textural parameters.

As shown in Table 3 , the R² value on the training set was generally greater than that on the test set, and the RMSE and MAE were generally lower than those on the test set. For M1, the GBDT and LightGBM models achieved the best prediction accuracy, with R² values of 0.71 each. RF had an R² value of 0.67, while CatBoost and MLP had the lowest prediction accuracies (R² ≤ 0.45). For M2, the overall prediction accuracy of the five models was greater than that for M1 (except for CatBoost). However, the GBDT and LightGBM models still achieved the highest prediction accuracies, each with an R² value of 0.76, while CatBoost and MLP had the lowest prediction accuracies (R² ≤ 0.40), with the highest errors. For M3, the combination of textural and morphological parameters did not significantly improve the prediction accuracy of the five models. According to Figures 7A–I , the error order is m2 > m1 > m3, and the R² value order is m3 > m2 > m1. With an increase in the richness of the indicators, the prediction accuracy increased, and the error decreased. The textural parameters led to better performance than the morphological parameters in estimating yield. Figure 7 shows the statistical results of cross-validation for soybean yield prediction based on the top-image parameters. M3 still performed the best overall, with the highest prediction accuracy for the GBDT model. Overall, in predicting soybean yield based on the top-image parameters, the best predictive model was GBDT, which was consistent with the conclusion in Section 3.2.

Building on the results from the previous sections, the side- and top-image parameters were combined to estimate the soybean yield. The three combinations of input variables were named S1-S3: S1 (m1 + M1), S2 (m2 + M2), and S3 (m3 + M3). The prediction accuracies of the CatBoost, RF, and MLP models improved compared to those presented in the previous sections. Notably, the MLP model demonstrated the most significant improvement, with higher prediction accuracy as the input indicators became more diverse, reaching a maximum of 0.74. The GBDT model also exhibited improved prediction accuracy and reduced errors compared to those in the previous sections, with RMSE = 3.12 and MAE = 1.99 on the test set. Moreover, the GBDT model consistently performed well. According to Figures 8A–I , the fusion of multiangle information contributed to reducing errors and enhancing the model prediction accuracy. In summary, the GBDT model is the optimal model for predicting soybean yield based on RGB image parameters. Figure 8 shows the statistical results of cross-validation for soybean yield prediction based on the side- and top-image parameters. The fusion of multiangle information helps reduce errors and improve the model’s prediction accuracy.

Based on the results presented above, GBDT emerged as the best predictive model, achieving the highest R² value with the S3 combination while maintaining relatively low RMSE and MAE values. To further explore the contribution of each indicator and achieve high predictive accuracy with minimal input parameters, an analysis of the importance of each indicator with respect to the S3-GBDT combination was conducted.

As shown in Figure 9 , the importance of the SME far exceeded that of the other indicators. This heightened importance could be attributed to its association with texture and photosynthesis (Heckmann et al., 2017). In addition to the SME, indicators such as SGD, SG, SDM, and TSC also exhibited considerable importance.

In the process of estimating crop yield based on remote sensing data, the selection of model variables is crucial (Ji et al., 2022). A comparison of the estimated soybean yield under the different numbers of input parameters ( Figures 10A–C ) revealed that the RMSE, MAE, and R² values stabilized when the number of input parameters was equal to or greater than 15. To achieve high prediction accuracy with the most lightweight set of input parameters, we identified the 15 most common indicators (6 side textural indicators, 6 side morphological indicators, 1 top textural indicator, and 2 top morphological indicators) as the final selection of predictive parameters.

Using these 15 indicators as input parameters for the five models, the prediction results are shown in Figure 11 . The prediction accuracy significantly decreased in the MLP model, while the other four models showed no significant changes. In fact, the GBDT model exhibited a slight improvement in R² values. These results further emphasize that the effective selection of input indicators can greatly reduce the computational load while maintaining the prediction accuracy. Filtering out irrelevant information is advantageous for constructing a lightweight model.

According to Table 4 , the broad-sense heritability of the 15 input parameters ranges from 0.68 to 0.84, with SME and SRA exhibiting the highest broad-sense heritability.

There has been extensive research on the use of RGB cameras to extract image parameters for predicting crop yields. However, previous studies have often had relatively narrow shooting angles, mostly capturing images from the top, leading to significant information loss. Moreover, in previous research, the focus was primarily on morphological indicators (Ma et al., 2022). reported a significant correlation between vegetation indices extracted from RGB images and cotton yield, suggesting that selecting optimal parameter features for cotton yield monitoring is feasible. However, from the perspective of model accuracy, there is room for improvement. Therefore, there is a need to extract more textural features or other color information from RGB images to enhance the model accuracy. Research by (Bai et al., 2022) also demonstrated that textural information has high potential for yield estimation. Building on previous studies, in this study, not only were the shooting angles expanded from the top and side, but three major categories of image parameters were also extracted—morphology, color, and textural parameters. This approach significantly enriched the variety of image parameters, providing a broader selection for identifying the optimal image indicators in subsequent analyses.

Indicator selection is a crucial step in machine learning. While algorithms determine the lower bounds of a model, input indicators set the upper bounds (Ma et al., 2022). The inclusion of a large amount of irrelevant input information can adversely affect model training, thereby impacting model accuracy. Therefore, a correlation analysis was initially conducted between all images and soybean yields, and only indicators with a Pearson correlation coefficient greater than 0.5 were obtained. This preliminary screening filtered out a significant amount of ineffective information.

After identifying GBDT as the optimal predictive model, further efforts were made to streamline the input indicators. Based on their importance, indicators were incrementally added to the model. As shown in Figure 10 , using 15 important parameters as input led to the desired results. With an increase in the input parameters, the RMSE, MAE, and R² values no longer exhibited significant changes, which might be due to the substantial collinearity among the input parameters, as the information coverage did not significantly improve with additional input indicators. Early analysis of input variables to reduce autocorrelation effects appears to have positive effects on model prediction (Han et al., 2019).

Moreover, an in-depth analysis of the final selection of 15 indicators, as shown in Figure 9 , revealed that the importance of SMEs was the highest, exceeding 30%. Previous research has indicated an association between textural parameters and plant photosynthesis (Heckmann et al., 2017). Our experimental design included two planting modes, soybean monoculture and strip intercropping, which magnified the impact of light conditions on soybean yield. Consequently, the importance of textural parameters became more prominent, further confirming our hypothesis. Additionally, the number of selected sideview image indicators far exceeded that of the top-view indicators. This dataset included 12 sideview image indicators (6 textural indicators and 6 morphological indicators) and 3 top-view image indicators (1 textural indicator and 2 morphological indicators). From Tables 1 and 2 , it is evident that predictions based on sideview images were better than those based on top-view images, and the selected indicators explain this predictive performance.

As a branch of artificial intelligence, machine learning technology has found widespread application due to its remarkable ability to integrate complex and dynamic biological knowledge with large-scale omics data (Shahhosseini et al., 2020; Van Klompenburg et al., 2020; Paudel et al., 2021). Machine learning technologies enable the construction of various prediction models and decision algorithms (Yan and Wang, 2022). As evident from Tables 2 , 3 , and 5 , the GBDT model achieved superior predictive accuracy compared to that of the other four models, consistently exhibiting the same pattern in terms of prediction errors. Therefore, the GBDT model is the optimal model for soybean yield estimation. The inherent advantage of GBDT, which is composed of numerous decision trees, lies in its continuous feature selection and partitioning process, which enhances its fitting capacity to the data (Zhang et al., 2023). GBDT achieves sample data classification by progressively reducing the residuals generated during the training process, significantly improving the data-fitting capabilities (Zhang et al., 2023).

The overall prediction performance of the MLP model is relatively poor. This is attributed to the model’s network structure, which includes multiple hidden layers, and each “neuron” is connected to all nodes in the preceding layer. This structure results in many parameters, making training challenging. However, with sufficient computational power and training data, the MLP performance can be significantly enhanced. This finding aligns with the results of our study, where an increase in the input parameters led to a noticeable improvement in the MLP prediction performance, reaching a maximum R² value of 0.74.

In this study, there are likely two reasons for the low predictive accuracy of the CatBoost model. First, due to the soybeans being grown in the field and influenced by field conditions, the growth status of each soybean plant is not very standardized (it may tilt, bend, break at the top, etc.; see Supplementary Figure S2 ), leading to many outliers. Although this study used the outlier test function in the car package to remove outliers, there is still a possibility of incomplete removal, which affects the predictive accuracy of the CatBoost model. Second, this study referred to the methods of predecessors and conducted preliminary screening of input parameters using absolute correlation (Yu and Liu, 2003; Cui et al., 2010; Ma et al., 2022; Li et al., 2023). However, it is possible that setting the screening threshold too low (Cor > 0.5) retained some indicators with weaker relationships, which had a certain impact on the predictive accuracy of the model. Therefore, in future research, more screening criteria should be added on the basis of correlations to enhance the initial screening of indicators. In the context of machine learning, heritability may not be a direct factor influencing algorithm selection, but it does affect the nature of the data and the structure of the feature space, thereby indirectly influencing the selection and performance of machine learning algorithms. In this study, the broad-sense heritability of soybean yield was 0.621. Therefore, when selecting machine learning algorithms, features with complex genetic mechanisms may have more diverse and nonlinear distributions. Algorithms capable of handling nonlinear relationships and complex patterns, such as support vector machines or decision trees, may be more suitable.

In general, as the richness of the indicators increases, the prediction accuracy improves, and the errors decrease. When using single-angle indicators as the input parameters, sideview indicators lead to significantly better performance than top-view indicators. When single-type indicators are used as the input parameters, the morphological parameters lead to better performance than the textural parameters from the sideview, while the textural parameters lead to better performance than the morphological parameters from the top view. We speculate that this phenomenon is due to the abundance of effective morphological parameters extracted from sideview images, whereas the top view provides a more intuitive view of young and tender soybean leaves. Therefore, sideview morphological parameters and top-view textural parameters are more critical for estimating soybean single-plant yield.

In this study, five machine learning algorithms were employed to estimate soybean yield. The results indicate that RGB images can be used to accurately estimate soybean yield. This research contributes to the identification of a high-throughput and nondestructive method for estimating soybean yield, accelerating the screening of germplasm and breeding materials.

Despite extracting as many indicators as possible in this study to estimate the final yield parameters, the information inherently contained in RGB images is limited, leaving room for further improvement in terms of the estimation accuracy. However, this study, which was based on data from 240 soybean varieties, two treatments, and two years of field data, benefitted from a large dataset, making our conclusions more applicable to real-world scenarios and highly credible. Future research could consider adding sensor types and enriching phenotypic parameters, thereby enhancing the prediction accuracy and reducing errors. Additionally, due to limited throughput and substantial workload, only images from the R6 stage were collected for analysis in this study. Subsequent research could involve image collection throughout the entire growth period to determine the earliest stage for accurate soybean yield estimation, further shortening the breeding process.

Finally, it is worth noting that selecting the best predictive model requires considering multiple factors comprehensively. Key factors include data characteristics such as scale, number and type of features, distribution, problem nature such as regression or classification, order or unordering, model characteristics such as interpretability, training speed, overfitting and underfitting, and resource constraints such as computation and time. For example, when dealing with many features, tree-based algorithms (such as GBDT and RF) may perform well because they can effectively handle high-dimensional data. For features with complex relationships, neural networks may be more effective. For regression problems, linear regression, decision tree regression, or GBDT may be good choices; for classification problems, logistic regression, support vector machine (SVM), RF, or deep neural networks may be more suitable. Additionally, it is important to evaluate model performance and select appropriate metrics through cross-validation. Therefore, there is no universally best model, and it is necessary to try multiple models and optimize them based on specific circumstances.