The research was conducted between June and October 2023 during the maize vegetation season at the 23-ha field, an agricultural site equipped with IoT sensors for data collection at Széchenyi István University in Mosonmagyaróvár, Hungary (32, 33). The area had an average annual precipitation of 580 mm in 2023, and an average annual temperature of 11.2°C, and during the maize vegetation season, the precipitation amount was approximately 400 mm with temperatures ranging between 18.5°C and 21.3°C. The soil types identified according to the USDA soil classification taxonomy (34) utilizing the soil texture triangle include loam, silt loam, and sandy loam (35) ( Figure 1 ). The terrain has a slight slope of 5% with elevation varying between 133 and 138 m (23), soil pH ranged between 7.12 and 7.8, and other soil properties are shown in Table 1 . Data collection from the field by sensors was carried out at 10- to 15-min intervals using LoRaWAN and Narrowband Internet of Things (NB-IoT).

A total of 405 soil samples were collected from depths of 5, 20, 40, 60, and 80 cm on three soil types: loam, silt loam, and sandy loam, with 135 samples each. Sampling locations were determined according to the USDA’s recommended procedures (36), ensuring representative and systematic coverage across the three soil textures, loam, sandy loam, and silt loam ( Figure 2A ). Each sampling point shown in the figure represents measurements taken at all five specified depths. The samples were collected using an auger on 10 different dates every 2 weeks (June 1–15, July 2–15, August 4–21, September 5–20, and October 3–18). They were collected from the field in the same sensor’s locations from a 1-m² circle around the sensor. The samples were saved in plastic bags to maintain their moisture content before being moved to the laboratory. After that, they were placed in pre-weighed containers and weighed using a digital scale to record their initial weights. The samples were then transported to the laboratory and oven-dried at 105°C for 24 h (37, 38). After drying, the samples were weighed again to obtain their post-drying weights. Finally, the empty weights of the soil moisture containers were measured. SMC based on dry weight was calculated using Equation 1 :

where:

The dataset was carefully prepared to address incorrect, missing, or hard-to-interpret values that could lead to overfitting or inaccurate, or deceptive results from the algorithm. Data preparation, including data transformation and statistical testing, was carried out using Python (version 3.10.12) (39) to ensure that there were no missing or zero values and to verify that the data types entering the model were accurate and consistent; in addition, basic statistical analyses including variation analysis and means calculations were conducted using Python. Additionally, dates and soil types were defined in the model to replace numerical values with appropriate date formats and soil type names. Following the data preparation, three ML algorithms—RFR, LSTM, and XGBoost—were employed to predict SMC in all soil types and depths. The dataset, which contained over 6,500 records collected from SMC measurements and meteorological data, was split into two parts: 80% for model training and 20% for testing. Each algorithm utilizes six inputs (SMC%, precipitation, humidity, temperature, solar radiation, and wind speed).

Four indicators were calculated to quantify the performance of the different models.

Mean square error (MSE): The MSE is the average of the squared differences between projected and observed. In other words, the MSE represents the variance of the mistake (43). It is calculated using Equation 2 :

where y_t is the ground truth, and y ^ represents the mean of the predicted values.

Root mean square error (RMSE): It captures the typical difference between predicted and observed values (43). It is calculated by taking the square root of the MSE. Unlike MSE, RMSE is measured in the same units as the original data, making it easier to interpret. Since RMSE considers squared differences, it emphasizes larger deviations from predictions. This makes RMSE valuable when minimizing significant discrepancies is critical ( Equation 3 ).

Mean absolute error (MAE) (43): It calculates the average of the absolute differences between predictions and actual observations ( Equation 4 ):

R-squared (R²): It is an indicator of statistical significance that compares the variation explained by the model to the total variance. Higher R ² indicates less difference between real and projected values. It is calculated using Equation 5 :

where y_i is the predicted value, y_t is the ground truth, and y ^ represents the mean of the predicted values.

Feature importance analysis was conducted for each ML model to evaluate and compare the contribution of studied meteorological variables to SMC prediction. For RFR, the Gini-based significance approach from scikit-learn was used to calculate the average reduction in error supplied by each feature across the forest (44, 45). In the LSTM model (an RNN model implemented in TensorFlow/Keras), feature importance is not inherently provided; thus, a model-agnostic permutation importance approach was applied and calculated by assessing the increase in MSE after shuffling each predictor variable in the validation dataset. Finally, with XGBoost, the built-in gain-based feature importance measure was utilized to calculate the cumulative improvement in the model’s accuracy when splitting by each feature (46). All analyses were conducted in Python and all feature importance results were computed separately for each soil type, depth, and model.

After analyzing 405 SMC% measurements in three types of soil and five depths over the maize vegetation season, the results shown in Figure 6 demonstrate that silt loam soil has the highest mean SMC, ranging from 14.99% to 18.27% due to its balanced sand, silt, and clay particle texture, which maximizes water retention (47), with the highest SMC (27.23%) recorded at 60 cm depth and the highest variability in the deeper layers (60 and 80 cm). The mean moisture content of loam soil varied from 12.61% to 16.19% at the five depths, with larger variability shown at deeper layers (60 and 80 cm), and maximum SMC value was at 20 cm depth with 22.1%. Sandy loam soil had a lower mean moisture content than loam soil due to its higher sand content, which decreased water retention, and the mean SMC ranged between 12.43% and 14.98%, with the highest SMC (20.77%) recorded at 80 cm depth. The low moisture content of sandy loam resulted in little variation among the five depths due to the high sand content, which reduces the water holding capacity and effect on soil water movement and dynamics through the soil layers (48). The highest SMC values and variability in deeper soil layers across all soil types are attributed to higher water retention capacity at depth compared with surface layers in the studied area (49). This variation in SMC% highlights the need to consider soil types and depth when monitoring soil moisture dynamics, as these factors had a significant impact on SMC variation (50). These results could be used in optimizing irrigation management, which will reduce water waste and enhance sustainability.

The performance of the three models (RFR, XGBoost, and LSTM) in predicting SMC in the studied soil types and depths resulted in several findings ( Table 2 ).

Among the three models, the RFR model consistently demonstrated the most robust and generalizable performance in all studied soil types and depths. For instance, in loam soil, at depths of 80 and 20 cm, the model achieved the lowest RMSE values of 0.89 and 0.98 and MAE values of 0.74 and 0.89, respectively. Figure 7 highlighted the model’s ability to highly accurately predict SMC at different layers in loam soil. In sandy loam soil, RFR performed best at depths of 20, 40, and 5 cm with RMSE values of 0.43, 0.51, and 0.54, respectively, and MAE values of 0.39, 0.49, and 0.51, respectively. Additionally, the model performed accurately in silt loam soil at 40- and 60-cm depths with RMSE values of 0.49 and 0.52, and MAE values of 0.42 and 0.41, respectively. These results align with the results of Cheng et al. (10) and Draper et al. (51) and suggest that RFR accurately captures the variation in SMC in all studied soil types and depths (52).

On the other hand, the LSTM models also showed promising results, particularly at shallow to moderate depths (20, 40, and 60 cm) and at all soil types where the meteorological factors have more impact on SMC variation at these layers, while it performed less consistently compared with RFR. For instance, in loam soil, at depths of 60 and 20 cm, the model achieved the lowest RMSE values of 0.39 and 0.80 and MAE values of 0.34 and 0.79, respectively. Figure 7 highlighted the model’s ability to accurately predict SMC at different layers in loam soil. In sandy loam soil, LSTM had the best performance at depths of 20 and 5 cm with RMSE values of 0.31 and 0.42, respectively, and MAE values of 0.22 and 0.38, respectively. In addition, in silt loam soil, the lowest errors were at 20- and 5-cm depths with RMSE values of 0.27 and 0.34, and MAE values of 0.21 and 0.36, respectively ( Figure 7 ), aligning with the results of Filipović et al. (3) and Park et al. (53). These findings indicate that the LSTM model demonstrates promising performance in predicting SMC, and the ability to accurately predict SMC, especially at shallower to moderate depths where the impact of environmental factors is higher than deeper layers (28).

The XGBoost models that offered a high performance in capturing complex and nonlinear relations performed well in different soil types and depths especially the shallower depths. The model showed a high accuracy at 40- and 60-cm depths in loam soil with the lowest RMSE values of 0.422 and 0.638, and MAE values of 0.354 and 0.507, respectively ( Figure 7 ). In sandy loam, the best errors were at depths of 40 and 5 cm where XGBoost had RMSE values of 0.099 and 0.159, and MAE values of 0.072 and 0.132, respectively, which considers higher performance in shallow depths compared with the results by Ren et al. (27), where the highest RMSE was 11.11 and MAE was 4.87. Similarly, in silt loam soil, it performed well at 5 and 20 cm with RMSE values of 0.249 and 0.256, and MAE values of 0.199 and 0.206, respectively. The acceptable threshold for RMSE value is 10% (54). During the experiment period, the models predicted SMC with RMSE values ranging from 0.24% to 0.9%. These low MSE and RMSE values, along with a high R² value, suggest that these models can effectively predict SMC as an alternative to classic empirical equations (55).

The variation in the measured parameters between deeper layers and upper layers is caused by different influencing factors. Shallower depths (5–20 cm) are directly affected by weather conditions such as precipitation and humidity, making it easier for the model to predict SMC accurately with high-quality data. On the other hand, deeper layers (40–80 cm) are influenced by factors like surface water dynamics, soil structure, and other soil properties, making it more complex for the model to accurately predict SMC in these layers (56, 57) and emphasizing the need to consider more features in model training for deeper soil layers.

The RFR exhibited consistent prediction accuracy with high R² values. In loam soil, RFR achieved R² values ranging from 0.820 to 0.971, with the highest R² at 80 cm depth. Similarly, in sandy loam soil, RFR performed well, with R² values ranging from 0.941 to 0.973, peaking at 20 cm depth. Moreover, in silt loam soil, the model had R² values ranging from 0.956 to 0.994, with the highest R² at 60 cm depth ( Figure 8 ). These results align with the findings of Cheng et al. (10) and Zhao et al. (58) and demonstrate that the RFR model offered the best generalization in the studied soil types and depths, especially the deeper layers where the impact of meteorological factors is less, and SMC is affected by other factors such as soil properties and components (11). On the other hand, the LSTM model showed promising accuracy results, especially at shallower depths where the model achieved R² values in loam soil ranging from 0.852 to 0.987, with the highest R² found at 60 cm depth. In sandy loam soil, LSTM performed well with R² values ranging from 0.964 to 0.996, peaking at 80 cm depth. Similarly, in silt loam soil, R² values varied from 0.982 to 0.980 ( Figure 8 ), with the best R² found at 60 cm depth; similar results were achieved in other research studies (28, 59). These findings indicate that LSTM models can accurately represent the temporal dynamics of SMC in various soil types and depths, particularly at shallow to intermediate depths, compared to other models (60). However, LSTM performance decreased as depth increased in all soil types.

In loam soil, XGBoost had R² values ranging from 0.835 to 0.983, with the highest R² at 40 cm depth, while in sandy loam soil, R² values ranged from 0.951 to 0.999, with the highest value at 40 cm depth. Similarly, in silt loam soil, R² values varied from 0.953 to 0.997, with the best R² found at 60 cm depth ( Figure 8 ). These findings indicate that XGBoost models may accurately capture the complicated interactions between SMC and environmental variables in a variety of soil types and depths. However, XGBoost’s performance decreased at deeper depths in loam and silt loam soils (27, 61). The results showed that the RFR model had the most consistent performance in all soil types and depths in comparison with XGBoost that performed best at shallower depths (5 to 40 cm) and LTSM in shallow to moderate layers (5, 20, 40, and 60 cm). In addition, RFR offered the best generalization capacity. These findings highlighted the need for further improvements for the models to increase accuracy in the deep oil layers where factors other than meteorological data have a significant impact on soil moisture dynamics.

The learning curves of the LSTM model indicated a decrease in loss values (MSE) with increasing epochs, particularly between 600 and 800, suggesting improved model generalization. However, there were slight variations in loss curves, indicating potential moderate overfitting at specific depths and soil types, such as 60 and 80 cm in loam soil ( Figure 9 ) and 20, 40, and 80 cm in silt loam soil ( Figure 10 ) (62). On the other hand, when the number of trees increased to 200, both the RFR and XGBoost models demonstrated decreasing loss values, showing improved model performance with increased complexity (63). However, deviations from this trend were observed, particularly in XGBoost models for sandy loam soil at 5 and 40 cm depth ( Figure 11 ) and silt loam soil at 20 and 40 cm depth ( Figure 10 ), where loss values slightly increased with the number of trees, indicating potential instability or poor performance under certain conditions. Overall, while all models showed improved performance with a higher number of epochs for LSTM (62) and greater model complexity for RFR and XGBoost (63), the LSTM model consistently outperforms and stabilizes in all studied soil types and depths.

The comparison between measured and predicted SMC% values at different depths in all soil types provides insight into the models’ predictive performance. The results show the model’s ability to accurately represent underlying patterns and variability in SMC at various depths and soil types. The close agreement between measured and predicted values indicates that the modeling approaches accurately predict SMC% (see Figure 12 ), and the mean measured and predicted values are provided in Appendix A. Compared with previous research by Alibabaei et al. (28) and Ren et al. (27), which developed different models to predict SMC, this research combines multiple soil layers’ SMC prediction with three AI-based ML models in three soil types, with the use of gravimetric data for more accurate validation and learning; the use of in situ IoT sensors data and high temporal resolution allows for more robust performance of the model than the use of only satellite images for model training. Thus, the soil-specific and depth-specific modeling achieved in this research is vital for precision irrigation management and drought monitoring. Variations between measured and predicted values, especially at certain depths or soil types, suggest many potentials for model development especially for deep soil layers (52, 55). Enhancing models’ generalizability and applicability could be done by considering extending the inputs with more variables such as different locations instead of only one location in this research, irrigations, evapotranspiration, vegetation indices from more vegetation seasons, and utilizing more soil properties such as soil texture and organic matter content.

The feature importance analysis revealed both common patterns and significant differences in how the three models prioritize the environmental variables. Overall, the studied meteorological data have contributed to SMC prediction, but solar radiation consistently showed as the most effective predictor, particularly for SMC in deeper soil layers (40 to 80 cm), achieving average values ranging from 0.4 to 0.85 across three soil types and five depths with highest values of impact on silt loam with three models (0.8 with RFR, 0.79 with XGBoost, and 0.7 in LSTM) ( Figure 13 ). This result demonstrates that evaporative demand caused by solar energy input is a dominating component driving soil moisture variability at depth, which is consistent with previous research indicating that solar radiation and temperature strongly influence soil water evaporation and drying (64, 65). On the other hand, precipitation had an important role in SMC prediction, especially at surface layer 5 cm in the three soil types with average values of 0.14 in RFR, 0.4 in XGBoost, and 0.25 in LSTM. These results emphasize the role of precipitation in increasing the SMC in surface layers (66). Humidity, temperature, and wind speed had varied importance among models, soil types, and depths. While analysis results showed the relatively high importance of humidity and wind speed in loam and silt loam soils, particularly at surface layers 5 and 20 cm with RFR and LSTM models, temperature had high importance value in sandy loam soil with the XGBoost model achieving values of 0.17 and 0.44 at 5- and 20-cm depths, respectively.

These variations represent each algorithm’s learning method. LSTMs, which are designed to capture temporal patterns and lag effects, may prioritize characteristics such as temperature that represent long-term environmental influences. RFR and XGBoost, on the other hand, use threshold-based decision rules and prioritize features that bring quick advances in predictive accuracy, such as recent precipitation or current wind speeds. Despite these model-specific variations, solar radiation and precipitation were the two most important variables impacting SMC across depths, soil types, and models. These findings are consistent with hydrological SMC principles, in which precipitation determines water input and solar radiation with temperature causing evapotranspiration and moisture loss (67). Overall, the feature importance results showed that the ML models effectively captured the key hydrological factors that influenced soil moisture dynamics. In particular, the ability to quantify and rank the impact of different predictors provides a solid base for optimizing irrigation systems. By identifying the key variables impacting SMC at certain depths and soil types, this study allows for data-driven decisions that support site-specific irrigation scheduling, reduce water use waste, and improve resource use efficiency. Additionally, feature importance analysis not only enhances model interpretability but also directly contributes to the objectives of sustainable water management and precision crop production. However, it is important to acknowledge that the influence of environmental predictors varies notably across soil types and depths. This highlights the need to develop soil-specific and depth-specific models that account for these variations by selecting key drivers tailored to each scenario. Additionally, the current study was limited to data from a single location and one growing season, which may affect the generalizability of the results to other regions or climatic conditions. Incorporating multi-season and multi-location datasets in future research would improve the robustness and transferability of the models. Furthermore, integrating intrinsic soil properties such as the percentages of sand, silt, and clay, along with other relevant variables like vegetation indices and organic matter into the predictive framework, could enhance model accuracy and adaptability. Expanding the comparison to include additional ML algorithms and larger datasets will also be essential to strengthen performance and ensure scalability for broader precision irrigation applications.