Accurate data collection is fundamental in machine learning algorithms. Besides diversity, data quality and representativeness are crucial considerations. Hence, constructing a high-quality dataset is paramount. This study manually collected and compiled diverse rock burst data from published sources, aiming to develop a highly accurate and reliable model. Various oversampling techniques were employed to enhance dataset quality.

To ensure dataset representativeness, various actual engineering rock burst cases were collected (Wang et al., 1998; Zhangjun et al., 2008; Afraei et al., 2019; Liu G. et al., 2023; Yu et al., 2013; Zhou et al., 2022; SUN, 2019; Feng and Wang, 1994; Mengguo et al., 2008; Xue-pei, 2005; Lai Feng, 2008; Zhang et al., 2011; Yu Xuezhen, 2009; Zhou et al., 2016), detailed in Supplementary Table S1. With a burial depth of 500 m as the delineation, the dataset comprised 69 rock burst cases, spanning hydroelectric power station construction, tunneling, and underground mining. Data collection mirrored the original approach (Yunzhang and Xuezhen, 2015). The input features included maximum tangential stress ( σ θ ), uniaxial compressive strength ( σ c ), uniaxial tensile strength ( σ t ), and elastic energy index ( W e t ), while rockburst intensity levels—categorized as None, Light, Moderate, and Strong—served as output prediction values. Violin plots in Figure 1 illustrate feature distributions, predominantly ranging from 20 to 80 MPa ( σ θ ), 80–180 MPa ( σ c ), 2–10 MPa ( σ t ), and 2–8 ( W e t ). Category percentages are depicted in Figure 2.

In real-world data scenarios, imbalances often arise where certain sample categories are underrepresented compared to others. This imbalance can cause some classifiers to favor results with more samples, thus exaggerating the accuracy of the model. But this may not accurately reflect reality. Correctly addressing sample imbalance is therefore crucial for improving model accuracy. Two main approaches are commonly employed: adjusting or integrating algorithmic models and reducing the sample number gap between classifications through sampling techniques.

The former approach involves analyzing the model and application cases comprehensively to make informed choices, albeit sometimes without satisfactory outcomes. The latter approach, favored by scholars for its simplicity and applicability at the dataset preprocessing stage, aims to bridge the sample number gap and enhance the realism of simulated data. Prior to this study, several popular oversampling techniques were outlined, providing the foundation for comparative analysis. See Supplementary Table S2 for details. These techniques provide diverse approaches to addressing sample imbalance and enhancing the realism and effectiveness of predictive models.

Suitable classifiers are the basis for building models with excellent performance. In this study, 12 mainstream classifiers were used. They are described in Supplementary Table S3.

The framework’s development process, depicted in Figure 3, entails several key steps.

(1) Initially, rockburst case data were collected from literature sources, revealing a deficiency in Strong class data.

To accurately gauge the performance of the six oversampling techniques across specific datasets and models, it is essential to employ appropriate evaluation metrics. In this context, the cross-validation method emerges as a suitable choice. Given the class-imbalanced nature of the collected dataset, the stratified 5-fold cross-validation score method is selected for evaluation. This approach involves dividing the entire dataset into five equally-sized subsets, ensuring that each subset includes representative data from all classes in the same proportion as the entire dataset. Moreover, random sampling of the data further enhances the model’s generalization ability. Figure 4 illustrates the specific process involved in implementing this approach. This methodology facilitates robust assessment of oversampling technique performance while accounting for dataset characteristics, contributing to more reliable model evaluation in real-world scenarios.

Once the rockburst prediction model was established, selecting appropriate metrics becomes crucial for assessing its performance. In classification models, accuracy serves as a key indicator, reflecting the model’s overall error rate. During model training, errors manifest in two forms: training error within the training set and testing error within the test set. While higher accuracy was desirable, it was imperative to prevent overfitting during training, ensuring robust performance on unseen data.

However, generalization error, occurring during testing on unseen data, was inherently unpredictable. To evaluate the model’s generalization performance, additional metrics were essential. Precision and recall emerge as complementary metrics, focusing on the model’s ability to correctly identify true positive instances. Precision quantifies the proportion of true positives among all predicted positives, while recall measures the proportion of true positives among all actual positives. These metrics assume particular significance in rockburst damage scale prediction, where accurately identifying hazardous conditions was paramount.

The F1 score, a harmonized average of precision and recall, offers a balanced assessment, capturing the trade-off between these two metrics. This score provides a unified metric that accounts for the model’s capacity to accurately predict positive instances while minimizing false positives. Utilizing these metrics, as depicted in Figure 5, ensures a comprehensive evaluation of model performance in this study, facilitating more reliable and accurate predictions.

The dataset analysis reveals a class imbalance, particularly in the Strong class. To address this, the dataset was expanded using six sampling methods: SMOTE, ADASYN, KMeansSMOTE, SMOTENC, BordenlineSMOTE, and SVMSMOTE, resulting in seven datasets including the original. These datasets are labeled as N1 to N6, respectively, with the original dataset labeled as OD (Table 1). SMOTENC and BordenlineSMOTE emerged as effective techniques for achieving balanced class distributions at the sample level, ensuring consistency across all four classifications. KMeansSMOTE notably increased sample numbers the most, while ADASYN and SVMSMOTE exhibited the smallest increases.

In addition to quantitative comparisons, Principal Component Analysis (PCA) (Abdi and Williams, 2010) is utilized as a dimensionality reduction technique. PCA aims to map N-dimensional features to K-dimensions while retaining the original high-dimensional features. Figure 6 illustrates this process, with the blue ball representing the original data and the red ball depicting the synthesized data.

Downsizing 5-dimensional data to two dimensions enables visualization of both the data distribution and the distribution of synthesized data.

To comprehensively assess oversampling techniques’ performance on seven datasets (OD, N1-N6), twelve machine learning algorithms (DT, ET, GBD, GPR, KNN, LGB, MLP, NBM, QDA, RF, SVC, XGB) were employed. The evaluation metric utilized was the 5-fold hierarchical cross-validation score, with default hyper parameter. Figure 7 presents the obtained data, revealing that among horizontal comparisons, the ET classifier + SVMSMOTE oversampling technique achieved the highest score, while the NBM classifier + original dataset attained the lowest, with a notable difference of 0.291. This underscores the significance of classifier selection. In longitudinal comparisons, varied sampling techniques exhibited distinct effects on classifiers. Notably, the GPR classifier displayed the highest discrepancy, with a potential difference of 0.1958 between selected sampling techniques.

To elucidate the impact of oversampling techniques across multiple models, Figure 8 presents the average scores of each technique. Notably, KMeansSMOTE demonstrates the highest average improvement across models, with a notable enhancement of 0.0998. This underscores the efficacy of KMeansSMOTE across diverse model architectures. Conversely, SVMSMOTE exhibits the lowest average improvement at 0.0178. Nonetheless, it is evident that employing oversampling techniques generally enhances model performance across various scenarios.

While various oversampling techniques were explored, the primary aim of the comparison study was to attain a highly accurate and reliable model. Consequently, the top-performing combinations—ET+SVMSMOTE and RF+SMOTENC—were selected for comparison. Additionally, the best-performing ET model from the original dataset was included for training and optimization.

Each dataset was randomly split in an 8:2 ratio, with 80% allocated to training and 20% to testing, ensuring models did not overfit (Zhang et al., 2024). Hyper parameter optimization was conducted using randomized 5-fold cross-validation, targeting key parameters—Estimators, Min samples split, Min samples leaf, Max features, Max depth, and Bootstrap. These parameters, being part of ensemble models built on decision trees, exhibited consistency across models. Subsequently, the training set was divided into five folds, with four utilized for model fitting and one for validation. Hyper parameter values were selected based on average accuracy across the five folds, as detailed in Table 2. These parameters were then used to evaluate overall classifier performance metrics—Accuracy, Precision, Recall, and F1 score—while other hyper parameter remained at default values.

The F1 score (Chicco and Jurman, 2020) is effective in incarnating the impact of class imbalance and serves as a key performance metric for classifiers. Comparison of the three classifiers with the test set prior to optimization, as depicted in Figure 9, reveals the significant impact of hyper parameter adjustments. Specifically, the combination of ET with the original dataset exhibits a notable improvement of 7.5 percentage points, while the pairing of ET with SVMSOTE demonstrates a substantial enhancement to 0.9375, compared to the original. Similarly, the combination of RF with SMOTENC notably improves by 18.4 percentage points. Notably, the ET with SVMSOTE combination attains the highest F1 score of 0.9375. Overall, hyper parameter tuning is crucial for achieving highly accurate modeling, underscoring its essential role in the process.

After hyper parameter adjustment, detailed evaluation of classifiers for overfitting or insufficient generalization ability is essential. The ideal model should exhibit high accuracy with minimal discrepancy between training and test sets. Figures 10, 11 provide visualizations for such assessments, with detailed data in Table 3. Overall, all three classifiers demonstrate sufficient accuracy post-hyper parameter tuning (Liu et al., 2024).

From the two-dimensional visualization in Figure 10, the ET and SVMSMOTE combination surpasses others in terms of generalization ability and test set accuracy, underscoring its superior performance.

Generalization ability is one of the main evaluation indexes for assessing the applicability of models. Good generalization ability can fully reflect the model’s ability to predict new data sets, and can greatly avoid the model overfitting, underfitting and non-convergence and other problems. In order to intuitively assess and compare the generalization ability of different models, Figure 10 is plotted. In this study, three combinations of models are evaluated together for their generalization ability, with the X-axis as the accuracy of the test set and the Y-axis as the accuracy of the training set, and the graph of the generalization ability of the three models evaluated is shown in the figure. Ideally, the model should show high accuracy both on the training set and the test set, with a small gap between the two. In this study, with this visualization, it can be clearly observed that the ET+SVMSMOTE model exhibits good generalization ability, highlighting the superiority of the data enhancement strategy.

Further analysis in Figure 11 reveals that the ET and SVMSMOTE combination outperforms in Accuracy, Precision, Recall, and F1 score. Notably, Accuracy improves to 0.9375, Precision to 0.9537, Recall to 0.9375, and F1 score to 0.9375.

In-depth exploration through Table 3 elucidates the superior performance of the ET and SVMSMOTE combination. Analysis by category reveals that SVMSMOTE effectively balances the number of Strong categories, thereby enhancing model performance in this category. Specifically, in the test set evaluation metrics, Strong category Recall improves from 0.6667 to 1, and F1 score from 0.8 to 0.8751. In the validation set, Strong category Recall improves from 0.8571 to 1, and F1 score from 0.9231 to 0.9653. This highlights the efficacy of oversampling techniques in achieving balanced datasets and subsequently improving model performance.

In contrast to prior studies, this research aims to develop an efficient modeling framework focusing on validating the resulting model’s performance. This model integrates the SVMSMOTE oversampling technique, Extra Trees integration method, and stochastic cross-validation of optimized hyper parameter to enhance reliability and accuracy in shallow rockburst prediction. Emphasis is placed not only on prediction accuracy but also on the oversampling technique’s significance in deeply analyzing various assessment indicators to ensure high reliability, accuracy, and generalization capabilities of the model. This contributes to safer, more reliable, and efficient underground engineering construction operations.

Furthermore, comparative analysis of different oversampling techniques across classifiers reveals their effectiveness in improving model performance by adjusting category numbers. Notably, KMeansSMOTE exhibits the most comprehensive improvement, increasing by 9.98 percentage points. However, at the individual model level, the ET+SVMSMOTE combination demonstrates the best performance, achieving a five-fold hierarchical cross-validation score as high as 0.7833. Additionally, this study conducts an in-depth analysis of the impact of SVMSMOTE oversampling on the ET classifier, validated through the analysis of category composition and evaluation metrics.

Table 4 presents a comparative analysis between the novel model proposed in this study and previous rockburst prediction models. Previous research indicates that while achieving high accuracy in modeling all rockburst is feasible, it remains challenging to further enhance accuracy to accommodate diverse stress state variations. Segmenting the rockburst dataset based on burial depth emerges as a crucial method for enhancing prediction accuracy. Remarkably, the ET+SVMSMOTE combination methods obtained through this approach consistently outperform mainstream model predictions in terms of accuracy.

A current limitation of this study is the inability to discern the interactions between oversampling techniques, hyper parameter tuning, and model performance. Future research endeavors should prioritize analyzing the significance of both factors on model interpretability, thereby enhancing overall model understanding. In addition, modeling all depths of rockburst occurrence based on previous studies could improve accuracy. However, it is critical to address the complex environmental factors associated with the occurrence of rockbursts at different depths. Future investigations should therefore distinguish between shallow, medium, and deep depths’ influences on rockburst prediction models and develop corresponding models to enhance accuracy and applicability.