The Windows 11 operating system was used to perform image processing tasks on a notebook with an AMD R7-7745HX and NVIDIA GeForce RTX 4060 with 8 GB VRAM. The algorithm in this paper was implemented in C++ using Microsoft Visual Studio 2022 as the integrated development environment. The flowchart of image processing is shown in Figure 1 . These steps will be described further in the following sections.

After the aforementioned preprocessing steps, the resulting image was nearly devoid of noise or outliers. Based on the resulting image, this study utilized the advantages of the RANSAC algorithm (Fischler and Bolles, 1981) to select a group of representative points with higher density. Subsequently, the least squares method shown in Equation 21 was applied to fit these points, avoiding the fitting line biased towards the density center of the points. The fitting effect is improved, as shown in Supplementary Figure S2C . Furthermore, as depicted in Figure 6C , it was worth noting that not all the pixels are required for navigation line fitting. The inclusion of additional pixels leads to increased calculative complexity. As a result, feature points that can capture the main characteristics of the crop should be selected before the fitting process. The aforementioned sliding window method is an effective technique to extract feature points. As shown in Supplementary Figure S2A , the data has been significantly simplified while effectively preserving the primary characteristics of the crop. In this case, the sliding window has a width of 16 and a height of 32.

The RANSAC is a widely used robust estimation technique that is critical in various computer vision and machine learning applications. The RANSAC algorithm can be expressed as shown in Equation 19. Two crucial parameters of RANSAC include the distance threshold and the number of iterations. The distance threshold can be chosen based on either empirical observations or experiments, and this study sets it to 0.155. The number of iterations is given by Equation 20.

where P = { ( x i , y i ) } i = 1 n represents the given set of points, L = { ( a , b ) } represents the parameter set for the line to be fitted, d is the distance from the point P to the line, L * is the optimal parameter for the line, | · |   represents the size of the set and t represents the distance threshold.

where C L represents the probability of the correct model, p = n i n l i e r s n i n l i e r s + n o u t l i e r s represents the probability of the correct model and n which is set to 2 represents the minimum number of points required for each model estimation. To enhance model accuracy, C L is set to 0.99 in this study and n i n l i e r s takes a value of 2 in the initial iteration.

where I = { p ∈ P | d ( p , a x + b ) < t } represents the optimal inlier set.

A set of crop images at different growth stages was selected for rigorous testing and evaluation to assess the efficacy of the aforementioned image-processing techniques. To quantitatively analyze the effectiveness of the image processing results, this study employed a manual annotation method to obtain ideal reference lines for crop row navigation in different images, and the obtained reference lines were regarded as a criterion. The reference lines were subsequently juxtaposed with the fitting lines, and the accuracy of the system was evaluated by computing the angular discrepancies between the reference lines and the fitted lines. The yaw angle between the fitting line and the reference line is calculated by the following Equation 22:

where P 1 → = ( x 1 , y 1 ) and P 2 → = ( x 2 , y 2 ) represent the vectors of two lines, respectively. To enhance the analysis of angular data, the following mathematical metrics for evaluation are employed: MEA, RMSE, and MRE, which are defined below.

where α ¯ and α i represent the MEA and the yaw angle between two lines of i − t h image, respectively, α i f i t t i n g and α i a n n o t a t i o n represent, respectively, the angles between the fitting line and the manually annotated line with the horizontal line at the bottom of the i − t h image. n is the total number of image samples.

The smaller the MEA is, the closer the fitting result is to the reference line. RMSE takes into account the size and distribution of the errors. MRE offers insights into the smaller the angle error relative to the reference line.

To facilitate effective user interaction and visual feedback, a human-computer interaction interface was designed and developed in this section, as illustrated in Supplementary Figure S3 . This interaction interface contains resolution selection, processing, processing time and result display. This interface serves as a critical bridge between the algorithm and its real-world applications. Once finishing the resolution selection and processing, the result can be shown in the interaction interface.

This section involves carefully curating a subset of sugarcane seedling images from ‘Code 12’ to ‘Code 19’ to serve as a validation dataset for assessing the efficacy of the image processing techniques on sugarcane. The yaw angles between the fitting lines and the reference lines are illustrated in Figure 7 .

Table 1 illustrates the evaluation of three growth stages, revealing an initial upward trend followed by a subsequent decline in MEA, RMSE, and MRE. This pattern can be attributed to variations in branch distribution. Specifically, during stages of ‘Code 12-13’, sugarcane seedling foliage is small and less dispersed, resulting in minimal data fitting dispersion and relatively good symmetry. However, during stages of ‘Code 14-16’, the foliage becomes more extensive and dispersed, leading to increased data fitting dispersion and diminished symmetry. Notably, in stages of ‘Code 17-19’, the foliage is large and evenly distributed, resulting in a high degree of data fitting dispersion but maintaining good symmetry.

This section provides a comprehensive evaluation of the image processing techniques applied to corn and rice seedling images, primarily focusing on the growth stages corresponding to ‘Code 12-15’ and ‘Code 16-19’, which aids in assessing the algorithm generality and applicability across crop image scenarios. The yaw angles between the fitting lines and the reference lines are illustrated in Figures 8 and 9 , respectively. The evaluation of corn and rice seedlings is denoted in Tables 2 and 3 , respectively.

Table 2 presents the evaluation results for two growth stages of corn seedlings. In the growth stage of ‘Code 12-15’, the algorithms exhibit enhanced performance on corn seedlings, with lower MEA, RMSE, and MRE compared to crops in the growth stage of ‘Code 16-19’. The primary factors contributing to this phenomenon are the morphological differences in corn during these two growth stages and weeds. As the growth period progresses, both the dispersion of corn leaves and the presence of weeds in the field can, to some extent, impact the accuracy of navigation line extraction.

Contrary to corn, the algorithms demonstrate better performance in the growth stage of ‘Code 16-19’ of rice compared to the growth stage of ‘Code 12-15’. The uneven dispersion of leaves in rice during the early growth stage has been well documented and attributed to the long length of the leaves. As the plant grows and more leaves are produced, the dispersion becomes more even.

Seedling less ridges are prevalent in agricultural cultivation (see Figure 10 ), especially for multi-year crops. Such ridges result in the wastage of land, fertilizers, and other resources, ultimately impacting crop quality and yield. Achieving precise navigation in rows affected by seedling-less ridges ensures efficient execution of tasks such as replanting and fertilization application.

As shown in Figure 10 , the navigation line can be extracted even in severe crop seedling absence in the crop rows. The MEA, RMSE, and MRE indicators in Table 4 also demonstrate that the extracted navigation line meets high precision requirements.

Based on the analysis in Section 2.2.3, the imaging method employed in this study identifies the baseline as the area with the highest pixel density along the vertical direction of the image, corresponding to the potential central crop row. By establishing the region of interest based on this baseline, the algorithm can dynamically classify crops belonging to the same row, unaffected by missing crops. The accurate extraction of the median crop row within the ROI enables the algorithm to effectively handle seedling absence. By comparing the distances between detected crop clusters and the baseline, the algorithm can robustly identify the correct row despite missing or sparse crop seedlings. This leads to improved accuracy and stability in scenarios with crop row discontinuities. These results demonstrate the centerline selection approach significantly enhances the robustness of crop row detection in challenging conditions.

The image processing techniques presented in this study demonstrate good generalization ability for corn and rice seedlings and perform well in extracting navigation lines in complex field environments, as shown in Figure 11 . The evaluation of diverse in-field environments is denoted in Table 5 .

As shown in Table 5 , it can be inferred that the algorithms are robust in dealing with images derived from complex in-field environments, especially for those images regarding illumination and disordered vegetation.

Compared to branch-and-leaf and soil environments, the algorithm in this study demonstrates better robustness under varying illumination conditions and in chaotic vegetation environments. This can be attributed to the algorithm primary focus on green pixels in the image, while morphological operations based on NZPR effectively eliminate redundant pixel information. However, in branch-and-leaf and soil environments, factors such as fallen leaves and the greenish appearance of soil in paddy fields introduce a certain degree of disorder to the green pixel information, resulting in a slight decrease in algorithm performance.

Data augmentation plays a crucial role in improving the robustness and generalization capability of machine learning models, especially when addressing variations in environmental conditions. In this section, the Albumentations library was utilized to simulate real-world scenarios under low-light and rainy conditions. This evaluation aims to further validate the algorithm adaptability to special conditions and enhance its performance in practical applications.

As shown in Table 6 , the navigation line extraction accuracy of traditional image processing algorithms exhibits significant differences under low-light and rainy conditions. According to the data, rainy conditions have a lesser impact on navigation line extraction, with lower MAE, RMSE, and MRE values for sugarcane, corn, and rice compared to low light conditions, indicating good algorithm robustness in rainy environments. In contrast, under low light conditions, the errors increase significantly, primarily due to insufficient lighting, which restricts the extraction of green features in the images. This is particularly evident in extreme low light environments, as illustrated in Figures 12A c, C c, E c, where green features are substantially suppressed. Although rainy environments also involve some degree of low light, the overall lighting is typically better than in pure low-light conditions. Among the crops, corn shows the lowest error across both conditions, while rice exhibits relatively higher errors, especially in terms of RMSE and MRE, highlighting the challenges faced by the algorithm in these scenarios.

In high-resolution image processing, the increase in the number of pixels necessitates the processing of a larger number of data points, which significantly augments the computational load and processing time. To address this challenge, code and algorithm optimizations were implemented. Ten images of three crop seedlings at different growth stages were used to evaluate the average time consumption of various modules in the proposed algorithm. As shown in Figure 13 , the processing time for a late-growth stage crop image with a resolution of 1920×1080 is largely dominated by the DBSCAN clustering algorithm. This is attributed to the high complexity of the neighbor search process, where processing time escalates markedly with increasing resolution. Based on Figure 13 , it is evident that the processing time of the optimized DBSCAN algorithm has been significantly reduced. At a resolution of 640×480, the processing time was nearly 30%. Meanwhile, at a resolution of 1920×1080, the processing time was reduced by almost 94%. However, it is worth noting that the processing time of the optimized DBSCAN algorithm still accounts for the majority of the total processing time.

To address the issue of DBSCAN, this study has implemented a KD-Tree to accelerate the neighbor search and utilized OpenMP for optimizing parallelization. Crop images with two resolutions (640×480 and 1920×1080) were utilized to test the processing time of the algorithm proposed in this paper, with 20 images of crops under different environmental conditions selected for each growth stage, as shown in Figure 14 . The average processing times for the crops at these two resolutions are presented in Table 7 .

It is observed that processing times for crops in the early growth stages (as exemplified by instance No. 1 in Figure 14A b and No. 20 in Figure 14B b) could be comparable to or exceed those recorded in later stages (as exemplified by instance No. 15 in Figures 14A b, B b). This discrepancy is likely attributable to varying field conditions; for instance, images from earlier growth stages may contain substantial areas of weeds.

A comparison between Figures 14A, B , as well as Figures 14C, D , reveals a significant reduction in processing time after optimization, with the improvement being particularly pronounced in high-resolution image processing. Although there is some variation in processing times across different crop images, the optimized algorithm exhibits much greater stability, indicating that the optimization not only reduces processing time but also enhances the stability of performance. However, even after optimization, the processing time for high-resolution images remains significantly longer than that for low-resolution images, highlighting the continued computational challenges posed by high-resolution data.

As shown in Table 7 , the evaluation suggests that a resolution of 640×480 offers an optimal balance for practical applications. Comparing the average processing times before and after optimization, for images with a resolution of 640×480, the average processing time decreased by approximately 25% to 30% after optimization. The optimization effect is even more pronounced for images with a resolution of 1920×1080, where the processing time was reduced by approximately 80% to 86%. The processing time was significantly reduced across all resolutions and crop types, indicating that the optimization methods employed, such as DBSCAN accelerated using KD-Tree and OpenMP parallelization, are effective in practical applications. As a result, the selection of resolution is decided by personalized needs.

This study proposes a crop row navigation line extraction algorithm designed based on the principle of prioritizing green pixels. The algorithm utilizes traditional image processing techniques, starting with a preprocessing stage that enhances green features and effectively removes noise through grayscale transformation and OTSU segmentation. Morphological operations guided by the NZPR significantly reduce unnecessary pixels in the image, isolating the main stem regions of the crop rows. Noise caused by the morphological operations is further removed using an eight-connected algorithm, which also refines the stem regions. During this step, vertical projection is applied to detect curve peaks (baseline), providing a foundation for clustering the central crop row. Optimal region detection is performed using DBSCAN enhanced with KD-tree acceleration and parallelization techniques, enabling efficient pixel clustering while reducing computational time. The baseline detection process helps identify potential target crop clusters, and the trapezoidal ROI ensures the algorithm focuses on relevant areas, even in cases of crop absence, while excluding irrelevant clusters. Sigmoid thresholding further segments the target crop rows, producing the optimal region for subsequent processes. Finally, the RANSAC algorithm combined with the least squares method is employed to fit the data points. This ensures robust accuracy even in the presence of outliers, successfully extracting the navigation line. Image preprocessing is a critical step in the navigation line fitting process. During this stage, the number of extracted pixels directly impacts the algorithm performance: too few pixels may fail to adequately represent the characteristics of the crop row, while too many pixels could significantly increase the computational burden in subsequent steps. Therefore, ensuring the extracted image reflects the crop row main features is a key objective in the preprocessing stage.

To evaluate the adaptability of this step under different environmental conditions, this study specifically selected low-light and rainy conditions as experimental scenarios, given their broader and more pronounced impact on image quality compared to other factors. As shown in Figure 15B , the total number of pixels extracted during preprocessing is highest under normal conditions, followed by low-light conditions, and lowest under rainy conditions. This indicates that the algorithm is affected to some extent under extreme environments. However, the extracted image pixels remain effective in representing the main features of the crop rows. Moreover, the navigation lines obtained under the three different conditions are nearly identical, demonstrating the proposed algorithm robustness and adaptability in extreme environments.

In this study, 640×480 and 1920×1080 are adopted as the target resolutions for image processing. These two resolutions reflect the consideration of mainstream resolutions in current visual devices and technology trends (Lin et al., 2020; He et al., 2022). The 640×480 resolution is widely used in real-time visual processing systems since that resolution requires reduced computational power, enable efficient algorithm operation within constrained computational resources. Conversely, 1920×1080, as a representative of higher definition resolutions, provides richer details and higher image quality, suitable for applications demanding higher detail accuracy, such as precise image analysis and advanced visual recognition tasks (Yang et al., 2020). By conducting experiments at these two resolutions, the adaptability of our proposed algorithm to different application demands can be validated. When applying our proposed algorithm to perform diverse field management tasks at the seedling stage such as weeding, spraying pesticides, and root-zone soil backfill, an optimal resolution can be selected through our developed human-machine interface in response to cross growth stage of crop seedlings.

Statistical analysis reveals that the algorithm average runtime is 0.31 seconds for images with a resolution of 640×480 and 0.51 seconds for images with a resolution of 1920×1080. In practical applications, agricultural robots performing field management tasks such as weeding, spraying pesticides, fertilizer application, and root-zone soil backfill typically travel at 0.6~1.5 meters per second. This indicates that the algorithm proposed in this study addresses the demands for real-time application.

The excellent adaptability of the algorithm to crops at different growth stages represents a significant breakthrough in this study. The average values of the growth stages of ‘Code 12-19’ from the evaluation results of Tables 1 – 3 show that MEA is 2.01°, RMSE is 2.95°, and MRE is 2.37%, which has demonstrated commendable performance in the case of cross-growth-stage crops. Additionally, by comparing the experimental results of different crops, it can be observed that the algorithms perform better on corn and rice seedlings, which have relatively concentrated and evenly distributed branches and leaves, than on sugarcane seedlings, which have unevenly dispersed branches and leaves.

An essential aspect of evaluating the performance and robustness of the proposed image processing techniques is the adaptability to complex field environments. Complex field environments, as depicted in Figure 11 , can pose various challenges for the algorithms, such as varying illumination ( Figure 11A ), fallen leaves ( Figure 11B a), dry branches ( Figure 11B b), soil background ( Figure 11C ), similar colors ( Figure 11D b) and occlusion ( Figure 11D a), dry fields (the first and second columns in Figure 11 ) and paddy land (the third column of Figure 11 ), different crop row spacing, etc. These factors can affect the accuracy and efficiency of the algorithms in detecting and segmenting crop plants from the images. Therefore, the algorithm adaptability to different complex field environments will be discussed in this section.

On the basis of the evaluation results of environments presented in Tables 4 and 5 , it can be inferred that MEA has an average value of 1.0018°, while RMSE has an average value of 1.446°. Moreover, MRE has an average value of 1.126%. Among the environments presented in Table 5 , the algorithms perform the best in chaotic vegetation, indicating that they can effectively handle situations where the branches and leaves of crops overlap and occlude with each other, while also accounting for the presence of offset crops and weeds. This remarkable performance can be mainly attributed to the proposed NZPR concept and the mapping relationship between NZPR and morphological operations. It is noteworthy that the common phenomenon of seedling less ridges in agricultural production can be successfully clustered into crop rows, thanks to the centerline selection method proposed in this study.

The inter-row spacing of the crops tested in this study was 1 meter for sugarcane seedlings ( Figure 7 ), 0.5 meters for corn seedlings ( Figure 8 ), and 0.3 meters for rice seedlings ( Figure 9 ). Upon analyzing the evaluation results of three crops, it becomes evident that the algorithms can effectively facilitate precise navigation for crops with inter-row spacing variability. As illustrated in section 2.2.3 and section 2.3.2, the utilization of vertical projection can successfully detect the baseline, which serves as the reference for centerline selection; crop clusters located in the centerline area can be determined by applying the distance threshold method, thus ensuring adaptability to inter-row spacing variability.

Arid land and paddy fields represent two major environments for crop cultivation, bearing considerable influence on agricultural production. For arid land crops, represented by corn, the average values calculated from Table 2 present that MEA is 1.39°, RMSE is 1.844°, and MRE is 1.665%. For paddy field crops, represented by rice, the average values calculated from Table 3 present that MEA is 2.3195°, RMSE is 3.4°, and MRE is 2.6145%. It is easily noticeable that the algorithm exhibits better adaptability to corn seedlings than rice seedlings. Apart from the influence caused by the inherent morphological differences between these two crops, factors such as reflections and glare in the paddy field environment can also interfere with the extraction of navigation lines. However, even in the case of rice seedlings, where the performance is relatively less favorable, there is a slight improvement in terms of MEA when compared to similar studies in the same category (Fu et al., 2023; Wang A. et al., 2021).

In practical agricultural operations, it is inevitable to encounter extreme conditions such as low light, nighttime, and rainy weather. Therefore, simulating these extreme environments, as shown in Figure 12 , through image augmentation is highly beneficial for further validating the generalization capability of the proposed algorithm. According to the results in Table 6 , the average values of MEA, RMSE, and MRE for the three crops under low light and rainy conditions are 1.41°, 2.67°, and 1.56%, respectively. When compared with the average values obtained under various real-world conditions in Tables 4 and 5 (MEA: 1.0018°, RMSE: 1.446°, MRE: 1.126%), it is evident that low-light and rainy conditions present greater challenges to the robustness and generalization capability of the proposed algorithm. This discrepancy can be attributed to the fact that lighting and rain affect the image globally, leading to diminished visibility of green features under such conditions. Since the algorithm proposed in this paper heavily relies on green information for subsequent processing, the decrease of these features in low-light and rainy environments directly impacts its performance. This highlights that future research could explore the use of sensors to acquire multidimensional data of crops, addressing the limitations of relying solely on image-based information.

In this study, traditional image processing methods and deep learning-based object detection methods exhibit distinctly different challenges in navigation line extraction. For traditional image processing, a significant source of error lies in the clustering process, particularly when crop rows appear closer together at the top of the image. This issue is exacerbated in cases where the inter-row spacing is narrower, as shown in Figure 18A, C with rice seedlings. The clustering algorithm may mistakenly group crops from adjacent rows into the central crop row, especially in the top regions of the image, leading to inaccuracies in navigation line fitting. A practical solution to this problem is to adjust the field of view of the camera. Since real-time navigation typically does not require distant views, narrowing the field of view can reduce the apparent convergence of crop rows at the top of the image, thereby mitigating clustering errors.

In contrast, deep learning-based object detection errors primarily stem from missed detections. As illustrated in Figures 18B, D , missed detections result in the loss of critical information from both the top and bottom regions of the crop rows in the image. When fitting the navigation line, the algorithm can only rely on a limited number of bounding boxes, significantly impacting navigation line extraction accuracy. In extreme cases, such as when only one bounding box remains for the central crop row, it becomes impossible to perform navigation line fitting, which is a critical failure for the algorithm. The reasons for these missed detections include the inability to extract features of the main stem. This issue can arise due to severe occlusion among crops, where overlapping leaves and stems obscure the main stem from the camera view. Additionally, in low-light conditions or during inclement weather, the features of the crops become less distinct in the images. This lack of clarity hampers the deep learning model ability to accurately detect and identify the main stems, leading to increased instances of missed detections. To address this issue, future research will focus on collecting more crop datasets under varying environmental conditions and using more efficient and suitable modules in the deep learning model.

This comparative analysis highlights that, in this study, traditional methods struggle with row spacing and clustering errors, and deep learning methods are vulnerable to detection omissions, especially in adverse environmental or crop conditions. Understanding these failure modes is essential for improving the robustness of navigation line extraction in practical agricultural scenarios.