The scour dataset used in this study was obtained from experimental studies conducted in a 2D wave flume, 22 m long, 0.6 m wide, and 1 m deep (Figure 1), at the University of Warwick’s Water Engineering Laboratory (Salauddin and Pearson, 2019b). The flume was equipped with a piston-type wave paddle, six Wave Gauges (WG) and active adsorption system capable of generating monochromatic and random waves, generating realistic sea states in the wave channel. The dataset consisted of over 120 experiments in which the scour characteristics at the toe of a sloping wall (1:2) with a shingle foreshore, of approximately 6 m length, on a 1:20 slope were observed and included a comprehensive range of incident wave conditions including both impulsive and non-impulsive waves. The JONSWAP wave spectrum with a peak-enhancement factor of 3.3 was applied to generate incident waves that were representative of the young sea state. The relative crest freeboard (R _c /H _m0 ), (where Rc is the crest-freeboard of the defence structure and H _mo is the wave height at the toe of the structure) ranged from 0.5 to 5.0 and this was achieved by applying six different types of toe water depths. The scouring characteristics were measured for both impulsive and non-impulsive wave conditions. The dataset comprising of 120 sets of observations, was split into a train-test set of 70%–30%.

For each test configuration, the scour depth was measured at the toe of the structure and at different locations along the wave flume in front of the structure. The maximum scour depth was then determined from these measurements. Analysis of the experimental data showed that, for the wave conditions tested, the maximum scour depth occurred at the toe of the structure. An insight into the database in terms of statistical correlation (Pearson R) revealed very low correlative relations between the scour parameters (described in the Glossary section) and the relative scour depth (=S _t /H _m0 ; where, S _t is the measured scour depth and H _m0 is the water depth at the toe of the structure). No negative correlation was observed between the variables, however, only R _c /H _1/3,deep and I _r showed a maximum correlation of 0.25 with the relative scour depth. Two kernel (ANN and SVMR) and two DT-based (RF and GBDT) algorithms were investigated in the study of Habib et al. (2023a) and it was reported that the algorithms performed satisfactorily in predicting wave overtopping at a vertical sea wall. The algorithms are hence also investigated for a scour dataset, since the intrinsic nature of the scour dataset is similar to what was applied in the overtopping study (Habib et al., 2023b).

The workflow followed in the data preparation, together with model development and testing is summarised in Figure 2.

SVMR is a category of supervised ML algorithms and an extension of the classification-based Support Vector Machines (SVM), typically employed for regression tasks (Noori et al., 2022). SVMR algorithms aim to minimize the prediction error and simultaneously maximize the margin around the fitting function, effectively identifying the best-fit function for a given dataset. Figure 3 illustrates a typical workflow structure for SVMR. For a regression problem using a training dataset containing interlinked input features (x _i ) and target values (y _i ), SVMR deduces a function f(x) that predicts the target values y based on input features x. The fundamental goal of SVMR is to construct a hyperplane that closely fits the training data within a specified tolerance of error margin (ε). Feature points inside the epsilon tube surrounding the hyperplane are regarded as support vectors, w, and do not incur any penalties. Points outside of this tube are penalized because they add to the error. The loss function is determined using Eq. 3: min 1 2 w 2 + C   ∑ i = 1 N ε i + ε 1 * (3) where, ε i and ε 1 * are slack variables that gauge how far the outliers are from the ε-tube, N is the total number of slack variables and C is a regularization factor that can be adjusted to determine the flatness of the hyperplane.

The main goal of the optimisation approaches related to SVMR is deducing the optimal values for w, and the slack variables, ε i and ε 1 * SVMR. The objective is to minimize the regularization term while ensuring that errors are within ε and slack variables remain non-negative. If non-linearity exists, the feature data is projected onto kernel space, a higher-dimensional hyperplane, which improves the model’s accuracy. The function k x i , x j defines the kernel space, and in this study, a Gaussian Radial Basis kernel function (RBF, Eq. 4) is utilised: k x i , x j = ⁡ exp   − x i , x j 2 2 σ 2   (4) where, σ is the kernel parameter. Gaussian RBF kernel is suitable for datasets with unknown or challenging-to-trace intrinsic feature characteristics (Roushangar and Koosheh, 2015). This is because the RBF kernel, based on the Taylor Series expansion, can accommodate an infinite number of feature dimensions. SVMR is particularly known for producing robust predictions when dealing with non-linear and high dimensional data (e.g., Kawashima and Kumano, 2017; Lan et al., 2023), similar to the dataset used in this study.

ANNs are well established in coastal engineering applications for tackling classification and regression tasks by mapping inputs to outputs, assigning weights to specific inputs, estimating and minimizing a loss function (example.g., Raikar et al., 2018; Verhaeghe et al., 2008; Zanuttigh et al., 2016; Formentin et al., 2017; EurOtop, 2018; Habib et al., 2023a; Habib et al., 2023b). Figure 4 illustrates the workflow of a feed-forward and back propagation ANN algorithm, including the input, hidden, and output layers. The input layer receives data from the training set. The information is only communicated to and from each layer within the neural network and not between neurones in the same layer. The model’s hidden layers are responsible for assigning numerical weights to the incoming information from the input layers and to the activation functions. The output layer of the network estimates the quantity predicted by the activation functions and then calculates the dependent feature(s) from the independent feature(s) in the input layers (Babaee et al., 2021; Khosravi et al., 2023).

A Multi-Layered Perceptron (MLP) ANN, which is feed-forward and back-propagation in nature, is adopted in this study (Figure 4). The term feed-forward and back-propagation essentially means that until a predetermined allowable error rate is achieved, the error rates are minimized by altering the loss functions through a combination of feed-forward (exchange of information from Input to Hidden to Output Layers) and back propagation (exchange of information from Output to Hidden and Hidden to Input Layers). The adjustment of weights and biases during the backpropagation stage is determined by the error rate. This process involves assigning new weights and activation functions to the hidden layers. The optimization of the number of hidden layers is usually based on the complexity of the input data, aiming to minimize prediction error (Elbeltagi et al., 2021; 2022).

RF and GBDT algorithms are typically categorised as Decision Trees (DTs). DTs are supervised machine learning algorithms used to predict an output variable (i.e., dependent or target variable) based on a set of independent variables (i.e., features). DTs are capable of tackling both classification and regression problems. In regression, they predict continuous or numerical output variables, while in classification, they predict class labels for discrete output variables (Yeganeh-Bakhtiary et al., 2022).

In the case of regression-based DTs, the training data is iteratively partitioned into rectangular regions, and the mean and median values within each region is estimated until a pre-determined stopping criteria are met. For example, given a training dataset, X = x 1 , y 1 , x 2 , y 2 . . . . , x n , y n , where x i represents an input feature vector for the ith training dataset and y i is the corresponding output, the DT algorithm divides X into a series of rectangular regions, denoted as R1, R2, R3, etc. For each region, the median and mean are estimated to serve as the prediction value P for that corresponding region. The final DT is constructed using the input features that distinctly divide these rectangular sections and yield the output variable with the smallest variance. DTs are commonly used in prediction tasks due to their ability to handle noise and non-linearity in input data independently (Pedregosa et al., 2011; Kotu and Deshpande, 2015; Yeganeh-Bakhtiary et al., 2023).

A Random Forest (RF) algorithm is an ensemble of DTs constructed from a random sub-set of training data. Figure 5 illustrates a schematic of methodological workflow for RF modelling approach.

RF model aims to reduce overfitting and enhance generalization by minimizing overexposure to any specific set of training data. The final prediction from tRF is the average of predictions made by individual DTs, often referred to as bagging. An additional advantage of RF is its capability handle both categorical and numerical data, further minimising overfitting.

The boosting strategy is another method for enhancing DTs’ predictive capabilities. An example of a Boosting approach is the GBDT algorithm (see Figure 6). The Mean Squared Error (MSE) between the predicted and actual values is measured in the boosting technique using a loss function. During training, the boosting algorithm aims to minimize this loss function by assigning numerical coefficients to input data, often through gradient descent. The GBDT algorithm, in particular, is known for rapidly minimizing the loss function, resulting in faster and more accurate predictions from DT models (Sutton, 2005).

To evaluate the performance of the machine learning algorithms in predicting relative scour depth, the predicted values were compared to the observed values using statistical metrics including the coefficient of determination (R ²), root mean square error (RMSE), mean absolute error (MAE), and relative absolute error (RAE). The Coefficient of Determination (Eq. 5) describes the percentage of the dependent variable’s fluctuation that can be predicted from the independent variables and, as such, serves as a gauge to evaluate the overall effectiveness of ML models (Cheng et al., 2014): R 2 = 1 − ∑ y i − y ^ i 2 ∑ y i − y ¯ 2 (5) where y i , y ^ i   a n d   y ¯ are the observed values, predicted values, and mean of all observed values, respectively.

The standard deviations between the observed and predicted values are reflected in the Root Mean Square Error (RMSE) calculated from Eq. 6, and discrepancies between these values, averaged across the number of observations, is expressed in terms of the Mean Absolute Error (MAE) as in Eq. 7: R M S E = 1 N ∑ i = 1 N q A − q P 2 (6) M A E = 1 N ∑ 1 N q A − q P (7) where, q_A and q_P are the actual and predicted relative scour depths, respectively.

In a regression test, the null hypothesis is that all of the regression coefficients are zero, i.e., the model is not predictive. The F-test is performed to determine whether accept or reject the null hypothesis. The F-test assesses whether the addition of predictor or dependent variables improves the model compared to a model with only an intercept (zero predictor variables). It quantifies the ratio of explained variance to unexplained variance (residuals) as (Eq. 8): F = S S R k S S E n − k − 1 (8) where, S S R = ∑ y i − y ^ i 2 , S S E = ∑ y i − y ¯ 2 , k and n are the numbers of independent variables and observations, respectively.

The plot of the residuals or the Discrepancy Ratio (DR) against the predicted values is also an important criteria about the relevancy of a prediction model and the residuals should ideally exhibit zero correlation with the predicted values (Sahay and Dutta, 2009; Salauddin et al., 2023).

The study of Kissell and Poserina, (2017) suggested that the statistical significance of regression models (where predicted values are compared against observed ones) should be holistically evaluated in terms of the r ² score, F-test score and the p-value. The values obtained from these statistical parameters should be in agreement to deduce the stability and accuracy of regression models.

The experimental dataset of Salauddin and Pearson (2019a) was deployed for training and testing of all the ML algorithms examined in this study following scalar transformations and feature selection. Training and testing of the models followed a common methodology which provided the basis for comparing modelled and measured dimensionless scour depths (S_t/H_{1/3 deep} [-]) in Figure 9. Results indicate that all the four ML-based models tested in this study are capable of providing realistic approximation of scour depths. In-depth statistical evaluation of the predictive models is presented in Table 2.

Notably, a number of data points for smaller (near to 0.0) relative scour depths fall outside the 95% CIs. This pattern is also evident for a few datapoints of large relative scour depth, while a few data points representing larger relative scour depths were inside the 95% CI zone. This suggests that while the algorithms were capable of robust overall predictions, but in the case of both smaller and larger relative scour depths, they exhibited some inconsistency. However, predictions for larger relative scour depths were more accurate (positioning on or very close to the regression line in Figure 9). The scatter in the graphs can be explained by the Pearson R score. Among the tested algorithms, ANN, GBDT, and RF showed similar scatter with relatively lower R scores compared to SVMR. SVMR, in particular, demonstrated comparatively more accurate predictions, as reflected by the highest r ² and R scores of 0.74 and 0.85, respectively. The RMSE values of the algorithms did not vary by a large margin with respect to one another. The SVMR yielded the lowest RMSE value of 0.28 while that of the RF was the highest at 0.33. The highest RMSE value was approximately 22% of the predicted maximum relative scour depth. From a computational efficiency perspective, under the given hyperparameter conditions (Table 1) and using a computer with an 8 cores CPU, 16 GB RAM, and 6 GB of dedicated GPU memory, the SVMR, ANN, GBDT and RF algorithms completed the prediction task (for the test set) in 2.5, 6.93, 14.83 and 22.3 s, respectively. This information suggests that SVMR outperforms the other algorithms in terms of computational efficiency.

Comprehensive statistical analyses of the developed ML-based models’ performance were conducted in this study. Statistical scores are then used to rank the performance of the four tested ML algorithms in predicting scour depth at sloping structures with shingle foreshore. Table 2 shows the results of performance evaluation of the algorithms according to the criteria outlined in Section 4.1.

The results from the evaluation metrics indicate that all the algorithms yielded strong r ² scores (r ² scores >0.40; Kissell and Poserina, 2017). Hence, the F-test was performed and it was observed that all the models yielded F-score higher than the critical F-test score of 4.15 (Table 2) and also the p-values for all the models were substantially (∼10⁻⁶) lower than the significance level of 0.05. These findings reflect the statistical significance of the results obtained from the ML algorithms and it can be inferred that the variations in the independent variable (actual relative scour depth) were accounted for by the dependent variable (predicted relative scour depth). The cumulative number of outliers in the models is expressed in the form of the RMSE. The SVMR algorithm yielded the predicted relative scour depth quantity with the smallest number of outliers, reflected in the lowest RMSE of 0.28 across all the tested algorithms. The RMSE of the other algorithms is not shown to differ significantly, suggesting the appropriateness and robustness of the proposed ML algorithms for predicting relative scour depth. A higher RMSE and MAE was coupled with lower r ² and vice versa for all the models. It is noted that the scale of MAE is dependent on the scale of the outputs (here, the predicted relative scour depth). The maximum and minimum absolute relative scour depth in the test set was 1.5 and 0.8, respectively, giving a mean relative scour depth of 1.15. The maximum MAE of 0.22 across the models was observed for the RF model. Conversely, the minimum MAE of 0.17 was determined for the SVMR model. Therefore, the range of MAE evaluated for this study was between 14.7% and 19% for the mean relative scour depth in the test set, consisted of 32 observations derived from the original set of 120 observations using a train-test split of 70%–30%. The significance of the MAE analysis is that the models were able to predict the relative scour depth with an approximate accuracy of 80%. Overall, the most accurate scour predictions were attributed to the SVMR model, with the least accurate predictions being associated with the RF and GBDT models, suggesting that DT based algorithms may be less suited for obtaining predictions from smaller datasets.

The method of evaluating the relative contribution of various features, also referred to as variables or predictors, in a predictive model is known as Feature Importance (FI). It is useful for selecting features, comprehending the underlying data, and getting new perspectives on the subject at hand. FI reveals which features have the most impact on the model’s predictions, essentially bridging the findings from ML to the physical consistency of the underlying processes (i.e., scouring in this study). The FI results are reported in two formats here, namely, the magnitude of the coefficients method and the permutation importance method. This is due to the fact that although the DT-based algorithms (i.e., RF and GBDT) had in-built FI analysis functions, the other two algorithms (SVMR and ANN) did not possess this function in Scikit-Learn’s module. In the magnitude of the coefficients method, the size of the coefficients directly reflects the significance of the feature. Greater absolute values imply greater significance of the predictors. The permutation importance method involves permuting a predictor’s values at random and analyzing the impact on model performance. The more performance is lost, the more significant the feature is thought to be. The results are reported in a similar format to that of the magnitude of coefficients method. Figure 10 summarizes the impact of the predictors on the prediction analysis.

In some experiments related to the measurement of relative scour depth at sloping walls with gravel foreshore, it was reported that the Iribarren Number I _r had a strong positive correlation with the measured scour depths for a given relative toe water depth (h _t /L _0m ) (Salauddin and Pearson, 2019b). Hence, it was expected that I _r would have the maximum influence in the prediction analysis to ensure consistency with experimental results. The FI analysis results show that 3 out of 4 (i.e., ANN, SVMR, and RF) algorithms identified I _r as the most important predictor. For the GBDT algorithm, I _r is ranked as one of the top three predictors, while the water depth at the toe of the structure (h _t ) is identified as the most important predictor. In Figure 10, the bars labelled as ‘others’, comprises the summation of the magnitude of importance of features including d ₅₀ , Duration, R _c , T _{m deep} , and L _m from the four tested algorithms. Therefore, it can be inferred from the results of FI, that the physical scouring processes are reasonably well-captured in the proposed ML-based models.

The residual plot for all of the tested algorithms is shown in Figure 11 . The residuals are independent of the predicted values, highlighting that the results are in good agreement regarding the reliability of the models.

An effective visual method to describe the statistical metrics from predictive models is the Taylor’s Diagram (Taylor, 2001). The Taylor’s Diagram (Figure 12) shows three statistical parameters, including the correlation coefficient projected as an azimuthal angle (in black), the radially plotted Centered Root Mean Square (cRMS) (in green), and the horizontally plotted standard deviations (in blue). Taylor diagram is particularly robust for assessing and comparing several performance aspectsof complicated models.

Due to the fact that they are both the square roots of squared differences between the actual and predicted values, standard deviation and the cRMS are comparable. However, they differ from one another in the context that RMSE is used to gauge the gap between actual values and the corresponding predictions while standard deviation accounts for the spread of data around the mean. The error of prediction, or the quantitative deviation is measured using the cRMS. Here, the cRMS of the SVMR is the lowest, while the standard deviation is the highest. This essentially means that while the predicted results are more spread across the regression line, the quantity of spread is small, indicated by the low cRMS score. Conversely, RF and ANN has lower standard deviation, but the quantity of deviation is high which is reflected by the higher cRMS score. The SVMR model also yielded the highest correlation coefficient of 0.96 followed by that of ANN (0.955), RF (0.95) and GBDT (0.92). Therefore, from a holistic point of view it could be inferred from the Taylor’s Diagram that the SVMR produced the more accurate values of predicted relative scour depth.