The study area was located near the Rãcãdãu River in Brasov County, Romania, at the geographical coordinates of approximately 45° 37′ 27″ N and 25° 45′ 46″ E. The location is covered by a mixed stand in which some old skidding roads were present, on which four sample plots (Table 1) were established. The site’s altitude ranges from roughly 700 to 720 m above sea level, and the dominant tree species found in the area are the European beech (Fagus sylvatica L.), Silver fir (Abies alba Mill.), and Norway spruce [Picea abies (L.) H. Karst]. During the data collection period, the weather was characterized by light to moderate rainfall and temperatures which varied between 6 and 11°C. Dry leaves and organic matter covered all the sample plots, which were chosen to reflect variations in terrain, slope, and rut depths (Table 1). The skid roads were very old and most likely they were used by cable skidders in the past to extract the wood from the area following selective felling. The ruts were exposed to weathering and erosion over time, resulting in different depths and shapes. The ruts varied in depth, from shallow to approximately 30 cm, while the slope of the plots was between 15 and 22° (Table 1 and Figure 1). The ruts found on the sample plots were not new, but the effect of time and weather factors since there was a long time from the last use of these skid roads for timber extraction. Nevertheless, these roads are still used by the local people seeking leisure in the area.

Data collection process was implemented in April 2022. The sample plots were rectangular in shape (Figure 1), measuring approximately 2 by 10 m and were geographically positioned using a handheld GPS receiver. Ten ground control points (GCP) were used in each plot, placed on their boundary and spaced at approximately 2 m each other (Figure 1), so as to form a grid overlapped on the boundary of each sample plot. GCPs were in the form of white plastic spheres, measuring 10 cm in diameter, attached to black stands, which were inserted vertically into the soil so as the top of each sphere was located at approximately 30 cm above the ground. These spheres were marked in advance on the sides with numbers from 1 to 10 by the use of a black permanent marker.

Two mobile LiDAR-based devices were used for scanning, namely the GeoSLAM Zeb-Revo (GeoSLAM Ltd et al., 2017) and the iPhone 13 Pro Max (Apple Inc., Cupertino, CA, USA, 2021). SiteScape and 3DScanner App were installed on the iPhone platform and used to collect the point clouds. The steps involved in the data collection process are illustrated in Figure 2. The scanning procedures used for the Zeb Revo were adapted to those used in earlier studies (Ryding et al., 2015; Gollob et al., 2021; Marra et al., 2022), meaning that the operator walked slowly around the edge of each sample plot, following a closed path while remaining about 1.5 m away from the boundary. This approach aimed to ensure that the entire area was scanned and to minimize position accuracy drifts and scanner range noise. After each scanning process and based on external control, the device automatically processed and saved the point cloud data on a USB stick. For the mobile phone and associated apps, scans were done at medium-density (MD), for which the clouds showed a good representation of the ground. The LiDAR sensor collected the 3D point cloud data as the operator walked along the plot’s axis. In each plot, two scans were taken by the Zeb Revo device; the iPhone platform was used to take two scans by 3D Scanner app, and another two by the SiteScape app. After each measurement day, the point clouds produced by both LiDAR platforms were transferred into a computer. Table 2 describes the point cloud datasets and their corresponding settings used in the C2C and data analyses.

Point clouds produced by scans had various sizes after pre-processing. Typically, the ZR files had between approximately 1.0 and 1.7 million points, SA between approximately 0.2 and 0.8 million points and SS between approximately 0.7 and 2.3 million points, respectively.

The results on the accuracy of low-cost mobile LiDAR scans using the C2C in the studied plots are summarized in Figure 5 and Table 3. Figure 5 gives only a limited amount of information on the accuracy comparisons. For more graphical information, see the Supplementary material. For the comparison of SS1 against ZR1 in Plot 1 (Figure 5), these metrics were computed on 47,372 points out of 50,000 subsampled from approximately 1.37 million sampling units of the SS test point cloud (Figures 5A, C). The results indicated that the mean distance was 0.010 m, with standard deviation of 0.009 (Figure 5E). Additionally, the final RMS errors during point cloud alignment and fine registration were 0.030 and 0.093 m, respectively (Table 3). For the C2C of SA1 against ZR1 point clouds in the same sample plot (Figures 5B, D, F), however, the metrics were computed on 46,828 points out of the 50,000 points selected from 337,086 sampling points of the SA test point cloud (Figures 5B, D). The results indicated that the mean distance was 0.004 m at a standard deviation of 0.005 (Figure 5F). Moreover, the final RMS error of the registration phase was 0.018 m, while the final RMS error obtained during the alignment phase was 0.042 m (Table 3).

Similar results on accuracy were obtained in the remaining C2Cs of SS and SA against the ZR reference point cloud. Table 3 provides a summary of all the results on accuracy of the LiDAR scans with SS and SA by considering the sample plot under question, RMS error obtained at the alignment and registration phases, as well as the mean and standard deviation of distance calculations between point clouds. As shown, the SA had lower RMS errors, mean distances and standard deviations compared to the SS, indicating that it may be more accurate in measuring distances with better repeatability. From these results, the range of final RMS errors of SS vs. ZR was 0.016–0.035 m (Range = 0.019 m). Additionally, the average final RMS error of SS vs. ZR was 0.023 m. In terms of the mean cloud-to-cloud absolute distance, the values varied from 0.004 to 0.039 m (Range = 0.035 m). The corresponding the standard deviation values varied from 0.003 to 0.081 (Range = 0.078). However, the average cloud-to-cloud distance and average standard deviation of SS vs. ZR were 0.016 m and 0.032, respectively.

For SA vs. ZR, the final RMS errors varied from 0.017–0.025 m (Range = 0.007 m). Moreover, the average final RMS error of SA vs. ZR comparisons was 0.020 m; the mean distance varied from 0.004 to 0.042 m (Range = 0.038 m). The corresponding standard deviation varied from 0.006 to 0.081 (Range = 0.075). However, the average cloud-to-cloud absolute distance and average standard deviation values of SA vs. ZR comparisons were 0.011 m and 0.021, respectively.

Overall, both SA and SS produced relatively accurate point clouds with low RMS errors, but SA appeared to have a slight edge in terms of consistency (repeatability) and smaller errors. The overall average final RMS error for SS was slightly higher than the overall average for SA (0.023 vs. 0.020 m). Besides, the SA has a slightly lower average mean distance and standard deviation compared to the SS, indicating that on average, the SA may be slightly more accurate than SS for cloud-to-cloud comparisons using the ZR as the reference point cloud. The results also suggest that the SA has less variability in its measurements than the SS and therefore greater repeatability by comparing the average standard deviations (0.021 vs. 0.032 m). The difference in accuracy between the two apps is relatively small, so it may not be a significant difference and thus suggesting that both apps are comparable in terms of overall accuracy.

Figure 6 and Table 4 show the results of the precision of low-cost mobile LiDAR scans. Figure 6 shows the results graphically only for the sample plot 1. The rest of the graphical results are given in the Supplementary material. The final RMS errors, mean distance, and standard deviation were used to evaluate the precision of the scans during both the alignment and registration phases. For instance, the comparison of SS1 and SS2 in Plot 1 (Figure 6) was based on 46,091 points out of 50,000 points subsampled from approximately 1.37 million of sampling units of SS test point cloud (Figures 6A, D). The results indicated that the mean distance was 0.006 m at a standard deviation of 0.009 m (Figure 6G). The final RMS error in registration was 0.017 m, whereas the final RMS error obtained during the alignment phase was 0.074 m (Table 4). Similarly, the statistical metrics for the C2C of SA1 and SA2 in the same sample plot 1 (Figure 6) were computed on 44,378 points out of 50,000 subsampled from approximately 0.83 million sampling units of the SA test point cloud (Figures 6B, E). The mean distance was 0.005 m with a standard deviation of 0.014 m (Figure 6H). Additionally, the final RMS error at fine registration was estimated at 0.012 m, whilst the final RMS error at the alignment phase was 0.039 m (Table 4). Regarding ZR1 and ZR2 C2C in the same sample plot 1 (Figure 6), the metrics were computed on 39,273 points out of 50,000 points selected from approximately 1.23 million sampling units of the ZR test point cloud (Figures 6C, F). The mean distance was 0.057 m, and the standard deviation was 0.185 m (Figure 6I); the final RMS error at fine registration was estimated at 0.019 m, while the final RMS error obtained during the alignment phase was 0.053 m (Table 4). A summary of all the comparisons is presented in Table 4.

The results presented in Table 4 show the final RMS errors at point cloud alignment and registration, and the mean C2C (cloud-to-cloud) absolute distances and their standard deviations of the point clouds generated by the iPhone 13 Pro Max (Apple Inc., 2021b) equipped with SS and SA and the professional mobile LIDAR platform of ZR for the four sample plots. From these results, the final RMS error values at fine registration when comparing ZR1 with ZR2 varied from 0.019 to 0.023 m (Range = 0.004 m). Additionally, the average final RMS error of ZR1 vs. ZR2 was 0.021 m. In terms of the range of values for the mean distance and standard deviation for ZR1 vs. ZR2, the mean distance varied from 0.005 to 0.201 m (Mean distance range = 0.197 m), and the standard deviation varied from 0.008 to 0.441 (Standard deviation range = 0.028). Similarly, the average cloud-to-cloud distance and standard deviation of these comparisons were 0.074 and 0.181 m, respectively.

For the comparison between SS1 and SS2, the final RMS error values varied from 0.015 to 0.017 m (Range = 0.002). The average RMS error of SS1 vs. SS2 was 0.017 m. Furthermore, the mean distance varied from 0.006 to 0.011 m (Mean distance range = 0.004 m), while the standard deviation varied from 0.009 to 0.037 m (Standard deviation range = 0.028); the average cloud-to-cloud distance was 0.008 m with a standard deviation of 0.023 m. Regarding the comparison between SA1 and SA2, the final RMS error values varied from 0.012 to 0.014 m (Range = 0.002 m). Moreover, the average RMS error of SA1 vs. SA2 was 0.013 m. Additionally, the mean distance ranged from 0.001 to 0.008 m (Mean distance range = 0.007), and the standard deviation ranged from 0.001 to 0.014 m (Standard deviation range = 0.012). However, the average cloud-to-cloud distance and standard deviation of these comparisons were 0.004 and 0.009 m, respectively.

Generally, these results suggest that the precision of the low-cost mobile LiDAR technology in estimating soil disturbance in forest operations is high. The final RMS errors, mean distances and standard deviations observed during the C2C comparison were generally low, indicating a high degree of precision. However, the results also show that the final RMS errors during the alignment phase were higher than during the registration phase. Overall, these results suggest that the SA app installed on the iPhone was the best option in terms of precision and consistency (repeatability).

This study aimed to estimate the accuracy and precision of the low-cost mobile LiDAR technology in estimating soil disturbance in forest operations using a cloud-to-cloud comparison approach (C2C), and on that basis, to give indications on repeatability. Regarding the metrics for estimating the accuracy and precision in this study, the RMS error and standard deviation are very similar statistical measures of variability (accuracy, precision, and repeatability) used in LiDAR research (Carter et al., 2012). However, studies show that the two values will be equal in a non-biased data set, when the error is normally distributed above and below zero (Carter et al., 2012). The closer the average RMS error values to zero, the better the performance, as it indicates a smaller deviation from the actual values (Girardeau-Montaut et al., 2005; Ahmad Fuad et al., 2018). Similarly, the lower the average cloud-cloud distance values, the better the accuracy and precision of the scanning device and technology used (Girardeau-Montaut et al., 2005; Ahmad Fuad et al., 2018). According to MacMillan et al. (2023), the standard deviation provides a sense of how close the complete collection of data is to the average value; data sets with modest standard deviations contain precisely organized data, while those with big standard deviations have data dispersed throughout a wide range of values. For example, if the standard deviation of a measurement is very small, it means that repeating that measurement will give similar results. Thus, lower values of the average standard deviation in this study indicate less variability in the precision and accuracy of the scanning devices and therefore greater repeatability.

In this study, the final alignment and registration RMS errors represent the error in aligning the SS and SA scan point clouds and registering them to the reference ZR point cloud, respectively. These errors were used to quantify the difference between test and reference point clouds, and subsequently estimate the accuracy and precision. These values gave us an idea of the variability of the registration and alignment accuracy and precision for each application and device. Nonetheless, the average standard deviation values showed how much the scanned data was spread out from the mean cloud-to-cloud distances, which were used to assess the repeatability of this method. Additionally, the mean cloud-to-cloud distance is the average displacement between the SS or SA point clouds and the reference model of ZR in each plot, which also gave us an idea of the accuracy and precision of the low-cost mobile LiDAR technology.

Overall, the performance of each application in terms of accuracy varied plot wise. SA consistently had the smallest final RMS error values, indicating that it was more accurate in capturing the geometry of the reference model. However, SA had considerably higher final RMS error values, especially in P4, which suggests that it may have some limitations in capturing more complex geometries (see Supplementary material). SS had the lowest overall mean cloud-to-cloud distance and standard deviation in P2. However, it had considerably larger final RMS error values in P1 and P4, suggesting that it is less accurate in capturing certain types of geometries. Similarly, based on the ranges of RMS errors, SA performed slightly better in terms of accuracy compared to SS. SA had a narrower range of RMS errors, which means that the errors varied less between different test point clouds. In contrast, SS had a wider range of RMS errors, indicating that its accuracy varied more between different test point clouds. However, SA’s RMS errors were generally smaller than SS’s RMS errors. SA’s lowest RMS error (0.017 m) was smaller than both of SS’s RMS errors, and SA’s highest RMS error (0.025 m) was still smaller than SS’s highest RMS error (0.035 m). This suggests that SA may be more consistently accurate across different test conditions, while SS may be less predictable in terms of accuracy.

In general, the registration error was smaller than the alignment error across all plots for all applications, suggesting that the registration process was generally more accurate. Overall, these results suggest that both applications can be useful for capturing 3D geometry of soil disturbance in forest operations. However, the accuracy of each application may vary based on the complexity of the soil geometry being captured as well as the complexity of space surrounding the sample plots. Further research could focus on identifying the limitations of each application and developing methods for improving the accuracy of 3D scanning.

Based on the average (mean) distance and standard deviation, SA had a slightly lower average mean distance and standard deviation compared to SS, indicating that it may be slightly more accurate. Overall, the average standard deviations for both apps were relatively low, indicating that they may have good repeatability on average. However, the standard deviations for some of the individual comparisons were relatively large (e.g., SS2 vs. ZR2 in P4 with a standard deviation of 0.081 and SA2 vs. ZR2 in P2 with a standard deviation of 0.016), indicating a lower repeatability in those cases. This suggests that while the studied apps may have good repeatability on average, there are some individual cases where the results could be less reliable. It is possible that this is due to differences in the underlying algorithms or hardware used by each application and device (Gollob et al., 2020, 2021). Further analysis, such as statistical testing or confidence interval estimation, could be used to better understand the variability of the results and their implications for the application being studied. Besides, comparing the average standard deviation values with the average RMS error values, it is evident that for all applications and plots, the standard deviation values were significantly lower. This suggests a good consistency and repeatability in the measurements, and a low uncertainty in the obtained results (Downing, 2004; Nakagawa and Schielzeth, 2010).

The overall accuracy and performance of each application and device may vary depending on the specific use, precision of the data, and other factors. Based on the results of this study, it appears that both applications can achieve a reasonable level of accuracy for cloud-to-cloud comparisons. Therefore, the choice of a given application may ultimately depend on other factors such as precision, user preferences, cost, and ease of use (Wang and Qi, 2021).

Similarly, the study found that the precision of SS, SA, and ZR point clouds was high. The ranges in the descriptive statistics indicated the variability in the final RMS error values, mean cloud-to-cloud absolute distances, and their standard deviation values for each application and device across all four plots; they shown that comparisons between SA1 and SA2 had the least variability in RMS error values, while the ZR1 vs. ZR2 comparisons had the highest variability. Moreover, the SA1 vs. SA2 C2C had the lowest range of values for both mean distance and standard deviation, indicating that the SA application was a consistent (repeatable) and reliable option. The ZR1 vs. ZR2 C2C had the highest range of values for both mean distance and standard deviation, indicating that ZR was the least consistent (repeatable) option. The SS1 vs. SS2 C2C falls somewhere in between with moderate performance in terms of precision and consistency (repeatability).

However, the range alone does not provide enough information about the overall precision; the average of the final RMS error values, mean distance and standard deviation may give a more accurate representation. Consequently, both the range and the average values were combined in the overall evaluation of the performance of the LiDAR platforms. From the results, SA1 vs. SA2 C2C had the lowest average RMS error (0.013 m), indicating that 3D Scanner app was the most precise option. The SA1 vs. SA2 had also the lowest average mean distance (0.004 m), and standard deviation value (0.009), indicating that 3D Scanner app was the most consistent (repeatable) option. On the other hand, the ZR1 vs. ZR2 C2C had the highest average RMS error (0.021 m), which suggests that it was the least precise option. Moreover, the ZR1 vs. ZR2 has the highest average mean distance (0.074 m), and the highest standard deviation (0.181), which suggests that this option was the least consistent (repeatable).

There is limited information available on similar studies for direct comparisons of these findings. For instance, SiteScape is supposed to enable users to easily take 3D scans that are accurate to within ± 1 inch, on average (Corke, 2021). Findings of this study are consistent with some previous research that also used LiDAR systems and/or C2C comparison methods to estimate the accuracy and precision of LiDAR technology. For instance, Milenković et al. (2015) evaluated the accuracy and potential of terrestrial laser scanning (TLS) for soil surface roughness assessment. Their study found that TLS provided highly accurate measurements of soil surface roughness with a mean difference of 0.52 mm between TLS data and ground truth measurements. Furthermore, Mikita et al. (2022) investigated the use of different types of laser scanning methods, including the iPhone 12 Pro Max LiDAR scanning apps, for assessing damage to forest road wearing courses. The study found that the root mean square error (RMSE) of the iPhone LiDAR scanning method was 0.023 m and the coefficient of variation (CV) for the vertical accuracy of the iPhone LiDAR scanning method was 0.25%. They compared these values to those obtained using a terrestrial laser scanner (TLS) and a mobile laser scanning system (MLS). They found that the RMSEs and CVs of the iPhone LiDAR scanning method were comparable to those of the TLS and MLS, indicating that the iPhone LiDAR scanning method can provide accurate and precise measurements. Additionally, the study suggested that the iPhone apps have the potential to streamline the data acquisition process and reduce costs compared to traditional terrestrial laser scanning methods. Similarly, Luetzenburg et al. (2021) evaluated the Apple iPhone 12 Pro LiDAR for its potential application in geosciences. The study found a mean horizontal error of 0.03 m and a mean vertical error of 0.05 m in a vegetation-dominated environment, making it suitable for high-resolution topographic mapping applications. Additionally, Jaboyedoff et al. (2009) used terrestrial laser scanning for the characterization of retrogressive landslides in sensitive clay and rotational landslides in riverbanks, finding that this method was highly accurate and precise for monitoring landslide movement and deformation. The study reported accuracy values generally less than 5 cm and precision values ranging from 1 to 2 cm. Ahmad Fuad et al. (2018) also found that the Iterative Closest Point (ICP) registration method and the Least Square Plane cloud-to-cloud distance approach were more accurate than other methods for spotting changes in 3D landslide surfaces utilizing Mobile Laser Scanning data between two periods.

In general, the quality of the scan would depend on various factors such as the type of soil, its moisture content, and the texture but when it comes to scanning soil, one important consideration is the resolution of the scanner (Jaboyedoff et al., 2009; Milenković et al., 2015). According to a study by Milenković et al. (2015), the accuracy of laser scanning for soil surface roughness measurement was affected by the point density of the laser scanner. TLS was also found to have high spatial resolution and could be used as a valuable tool for monitoring soil erosion and other environmental changes (Milenković et al., 2015). Similarly, a study by Jaboyedoff et al. (2009) found that increasing the point density resulted in a significant reduction of the error in comparisons between terrestrial laser scanning datasets which was approximately 3 to 6 cm in their study. Therefore, a higher resolution scanner is likely to provide more accurate results when scanning soil surfaces. Another factor that could have affected the accuracy and precision of the low-cost LiDAR scans in this study was the presence of vegetation or other objects on the soil surface (Salmivaara et al., 2018). Several studies indicated that the presence of small vegetation and litter on the soil surface caused errors in the determination of soil surface elevation and roughness, compaction and rutting using LiDAR and TLS methods (Milenković et al., 2015; Salmivaara et al., 2018; Mohieddinne et al., 2022). According to Magtalas et al. (2016), the meter level distances and standard deviations in their study were produced by the fact that both point clouds’ whole contents including vegetation were used to calculate the cloud-to-cloud distance when they employed distance computations to further compare their findings. However, the descriptive statistics they used to assess accuracy and precision reduced when the C2C was performed on non-vegetation point clouds using the same procedure. Thus, the vegetation growing on the soil’s surface, which exhibited elevation differences of a few meters between the two datasets, might be the source of the meter level deviations (Magtalas et al., 2016). The accuracy and precision values from their study cannot, however, be directly compared to those from this study, because their technique of assessing the accuracy and precision differs from that used herein. Thus, further research is necessary to explore the effects of vegetation on the accuracy and precision of these low-cost LiDAR systems.

Despite being a useful method for analyzing LiDAR data, the C2C has important limitations in its use (Girardeau-Montaut et al., 2005; Cheng et al., 2018; Kharroubi et al., 2022). One of the main limitations is the assumption that the point clouds being compared are of the same geographic location. The comparisons may be inaccurate if there are any variations in the position or orientation of the point clouds during scanning (Girardeau-Montaut et al., 2005; Lague et al., 2013; Kharroubi et al., 2022). Additionally, the accuracy and precision of the cloud-to-cloud comparison method depend on the quality and resolution of the point clouds generated by the LiDAR systems (Cheng et al., 2018; Kharroubi et al., 2022). The C2C method is not robust to changes in point density and point cloud noise (Girardeau-Montaut et al., 2005; Lague et al., 2013), which may affect the accuracy and precision of the results. The accuracy of the method can be reduced if the clouds from different LiDAR systems have different point densities (Cheng et al., 2018; Kharroubi et al., 2022). These problems were addressed in this study by modeling the soil surfaces locally to prevent problems with density change.

Moreover, this study is constrained by the small sample size because it only included four rectangular plots, each measuring around 20 m². More studies may be necessary to evaluate the suitability of this technology in different forested landscapes. Furthermore, the studies cited in this research paper are case studies and may not be generalizable to other locations or forest types. As such, additional research may be necessary to test the applicability of low-cost mobile LiDAR technology in other forested areas. In addition, the cost of high-end LiDAR systems for soil disturbance estimation may not be justifiable for small-scale forest operations. The potential benefits of low-cost mobile LiDAR technology for these operations may be limited by their accuracy and dependence on mobile platforms.

Numerous studies demonstrate that attaining accurate results with low-cost LiDAR technology depends on several variables, including the calibration parameters of the LiDAR platform components, the underlying point cloud density, the scanning settings, and the data processing methods used. Therefore, it is necessary to keep exploring new low-cost LiDAR technologies as they appear. Future studies might consider integrated low-cost LiDAR systems that can potentially offer enhanced capture options with a greater spatial resolution and a longer effective range in forest environments. Additionally, increasing plot coverage while maintaining low operational costs can potentially be achieved by designing and implementing a multiplatform LiDAR sensor solution using Apple’s iPhones. For instance, the use of selfie sticks in future research can potentially increase plot coverage and point cloud capture. Moreover, incorporating a multiple-iPhone strategy for the capturing of LiDAR point clouds may be able to shorten the time required for data collection. However, future studies should assess how quickly (time efficiency) point cloud data can be captured and processed using low-cost LiDAR technology. Besides, it will be necessary for extra caution to be taken in using the point picking tool in CloudCompare to ensure that the exact centers of the GCPs are hit to ensure proper alignment and subsequent registration of the point clouds.

Additionally, both the procedures and protocols used in low-cost LiDAR scanning and the experience of the operator are relevant factors that can impact the quality (accuracy, precision, and repeatability) of the resulting LiDAR data (Rathore, 2017). Several studies suggest that that optimizing the scanning procedure, by carefully planning the survey area, selecting appropriate scanning parameters and designing efficient scan paths, choosing the scanning pattern and processing of the data collected are essential steps in the process and can improve the quality of the data while reducing costs (Rathore, 2017; Wang and Menenti, 2021). The experience of the operator is also critical in ensuring the quality of data collected by low-cost LiDAR systems. An experienced operator can anticipate problems that may arise during the survey and adjust the scanning plan accordingly. Additionally, an experienced operator can make a significant difference in the accuracy of the data collected, particularly in areas with dense vegetation cover by understanding the limitations of the technology and taking appropriate corrective actions.