This study included 60 EGFR-T790M-positive NSCLC patients with brain metastases (115 brain metastases in total) who received second-line osimertinib treatment from West China Hospital. The overall study design is detailed in Figure 1. This retrospective study was approved by the Biomedical Research Ethics Committee of West China Hospital, Sichuan University, and patients’ written informed consent was waived.

The main inclusion criteria included: (1) patients with advanced NSCLC diagnosed by histopathology and imaging examinations, classified as stage IV according to the American Joint Committee on Cancer (AJCC) 8th edition TNM staging system; (2) patients previously treated with first-line EGFR-TKI therapy (first/s-generation EGFR-TKI) who had progressive disease (RECIST 1.1 criteria); (3) patients with EGFR-T790M mutation after first-line EGFR-TKI therapy and received sequential second-line osimertinib treatment; (4) patients had contrast-enhanced brain MRI images within 1 month before osimertinib treatment; (5) patients had ≥1 measurable brain metastases before osimertinib treatment (RECIST 1.1 criteria). Main exclusion criteria of cohort 1: (1) patients with missing or poor-quality brain MRI images; (2) patients with leptomeningeal metastases or brain metastases with unclear margins; (3) patients with incomplete clinical data; (4) patients who received other anti-tumor treatments such as radiotherapy and chemotherapy during osimertinib treatment. Data collection was from September 2015 to November 2021. The last follow-up was in December 2021.

Patients included in the study were scanned with one of the following 5 MRI machines: Siemens Avanto (1.5 T), UIH UMR588 (1.5 T), TOSHIBA MRT200SP5 (1.5 T), GE discovery MR750W (3.0 T) and Philips achieva (3.0 T). All patients had brain MRI contrast-enhanced T1-weighted (T1CE) images. In line with previous studies, MRI radiomics features were extracted based on T1CE images in the current study (24, 29, 30). The acquisition parameters of T1CE images are as follows: Siemens: repetition time/echo time (TR/TE) = 2040/3.94 ms, flip angle = 15°, matrix = 256 × 224, slice thickness = 1 mm, pixel size = 1 × 1 mm²; UIH: TR/TE = 13.83/5.9 ms, flip angle = 10°, matrix = 256 × 232, slice thickness = 1 mm, pixel size = 1 × 1 mm²; GE: TR/TE = 2383/20.78 ms, flip angle = 142°, matrix = 320 × 256, slice thickness = 5 mm, pixel size = 0.4688 × 0.4688 mm²; TOSHIBA: TR/TE = 580/8.0 ms, flip angle = 90°, matrix = 320 × 160, slice thickness = 6 mm, pixel size = 0.6875 × 0.6875 to 0.75 × 0.75 mm²; Philips: TR/TE = 130/2.3 ms, flip angle = 80°, matrix = 256 × 205, slice thickness = 6 mm, pixel size = 0.4492 × 0.4492 mm². After rapid intravenous injection of gadolinium-DTPA contrast agent (dose 0.1 mmol/kg, flow rate 2–3 mL/s), enhanced brain MRI scans were performed, and T1CE images were obtained.

Two radiologists evaluated the MRI morphological manifestations of brain metastases, and disagreements were solved through discussion. Both radiologists were blinded to patients’ treatment responses to osimertinib. MRI morphological manifestations include: (1) the number and largest diameter of brain metastases; (2) the location of brain metastases (frontal lobe, parietal lobe, occipital lobe, temporal lobe, cerebellum, and others); (3) the presence or absence of edema around brain metastases (peritumor edema); (4) enhancement pattern (with or without ring enhancement).

This study retrospectively collected clinical data of all patients, including age, gender, smoking history, Eastern Cooperative Oncology Group performance status (ECOG-PS) score, clinical TNM stage, initial EGFR mutation, first-line EGFR-TKI agent, and patients’ progression-free survival (PFS) in first-line EGFR-TKI therapy. The clinical stage was assessed according to the AJCC 8th edition TNM staging system. PFS of first-line EGFR-TKI therapy was defined as the interval from initiation of treatment to radiographic progression (RECIST 1.1 criteria).

Intracranial efficacy of osimertinib on each individual brain metastatic lesion was assessed according to RECIST 1.1 criteria. Intracranial objective response (favorable intracranial treatment responses) included complete response (CR) and partial response (PR), while poor intracranial treatment responses were stable disease (SD) and progressive disease (PD). To determine if radiomics features can predict unfavorable therapeutic responses in patients undergoing Osimertinib treatment, we selected SD/PD as the endpoints, and the secondary endpoint was intracranial progression-free survival (iPFS) which refers to the interval from the start of osimertinib to intracranial tumor progression.

The brain MRI images obtained by the Picture Archiving and Communication System (PACS) were uploaded to the open-source software 3D slicer in DICOM format (Version 4.10.2). ¹ Two radiologists independently and manually outlined the VOI of brain metastases on the axial T1CE images. Up to five brain metastases were selected as target lesions in patients with multiple brain metastases (31). If the number exceeds five, the largest five brain metastases were selected for delineation (32).

The image preprocessing and radiomics feature extraction methods applied in this study are in line with the recommendations of the Image Biomarkers Standardization Initiative (IBSI). ² To minimize differences in image acquisition parameters between different MRI scanning machines and improve reproducibility, we performed image preprocessing before radiomic feature extraction, mainly including five steps: (1) Use the “Swiss Skull Stripper” module in the 3D slicer software to perform the skull stripping of the T1CE image. (2) Use the “N4ITK MRI Bias correction” module to perform bias field correction on the image to eliminate the inhomogeneity of low-frequency intensity. (3) To standardize the voxel spacing, the sitkBSpline interpolation algorithm is used to resample all MRI voxels (1 × 1 × 1 mm³). (4) The image signal intensity is normalized by grayscale discretization (bin width = 25HU) uniform processing. (5) Apply Wavelet and Laplacian of Gaussian (LoG) filter transformation (sigma: 1.0–2.5) to remove interference signals.

After image preprocessing, the Python package “PyRadiomics” (33) ³ embedded in the 3D-slicer software was used to extract radiomic features from the VOI of each delineated brain metastatic lesion. Totally, 1,223 radiomic features were automatically extracted, including 107 original features, 744 Wavelet features, and 372 LoG filtered transformation features. These radiomic features were grouped into three categories: 14 shape features, 234 first-order features, and 975 texture features. Texture features included gray-level cooccurrence matrix (GLCM, N = 312), gray-level run length matrix (GLRLM, N = 208), gray-level difference matrix (GLDM, N = 182), gray-level size zone matrix (GLSZM, N = 208), and neighborhood gray tone difference matrix (NGTDM, N = 65).

After extracting the radiomic features, to minimize the multi-machines effect, we performed the ComBat harmonization (34), ⁴ a widely used algorithm to correct the batch effect of radiomics extracted by different scanners (35). We also applied Z-score transformation to normalize the radiomic features.

To evaluate the consistency of the radiomic features extracted by the two researchers, we calculated the inter-observer interclass correlation coefficients (ICC) for each radiomic feature. An ICC value>0.9 was considered as the criterion of high reproducibility. The preliminary selection of predictive radiomics features consisted of two steps: (1) univariate logistic analysis was used to initially screen out potential predictors; (2) considering the high degree of collinearity within radiomic features, which may lead to overfitting of the radiomic model, we performed a correlation analysis on the radiomic features and excluded redundant features through the findCorrelation function in the “caret” R package.

A total of 115 measurable brain metastases were randomly split into the training (n = 80) and validation (n = 35) groups (ratio: 7:3). To reduce the bias from cluster correlation, we assigned the multiple brain metastases from the same patient simultaneously to either the training or the validation group as the previous study did (24, 36).

Eight feature selectors and eight classifier machine learning algorithms were used to construct radiomic models for predicting the intracranial efficacy of second-line osimertinib therapy. A nested 5-fold cross-validation strategy was applied. Eventually, a total of 64 (8 × 8) combinations of radiomic feature selectors and classifiers were generated.

In selecting the feature selectors, Boruta was chosen for its comprehensive approach to identifying all relevant features, ensuring that no potential predictive feature is overlooked. Minimum Redundancy Maximum Relevance (mRMR) was included due to its balanced approach in selecting features that provide the most information while minimizing redundancy, which is crucial for enhancing model performance. LASSO was selected for its dual functionality in variable selection and regularization, which helps in managing overfitting and improving the model’s generalizability. Relief was chosen for its ability to capture feature dependencies and interactions, which is often beneficial in complex datasets. InfGain and GainRatio were utilized for their effectiveness in measuring the importance of features based on information gain, which is essential for building robust models. Gini was included for its simplicity and proven effectiveness in feature selection tasks. DistEuclid was chosen for its straightforward approach in selecting features based on Euclidean distance, aiding in identifying the most relevant features for classification tasks.

Regarding the classifiers, the artificial neural network (ANN) was chosen for its powerful ability to model complex nonlinear relationships in data, making it suitable for the intricate patterns found in radiomic data. Adaptive boosting (Adaboost) was selected for its capability to improve the performance of weak classifiers, thereby enhancing the overall predictive accuracy of the model. Fast nearest neighbor (FNN) was included for its efficiency and effectiveness in classification tasks, particularly when dealing with high-dimensional data. Gradient boosting decision tree (GBDT) was chosen for its strong performance in handling structured data and its ability to produce high-accuracy models through iterative boosting. Naive bayes was selected for its simplicity and effectiveness in probabilistic classification, making it a robust choice for many classification tasks. Support vector machine (SVM) was included for its effectiveness in high-dimensional spaces and its ability to construct hyperplanes for classification tasks. Random forest (RF) was chosen for its robustness in handling large datasets with high dimensionality and its ability to prevent overfitting through ensemble learning. Stepwise logistic regression (SLR) was included for its simplicity, interpretability, and effectiveness in linear classification tasks.

The main R packages involved were as follows: “Boruta,” “mRMRe,” “e1071,” “naivebayes,” “CORElearn,” “ANN2,” “fastAdaboost,” “FNN,” “gbm,” and “glmnet.” The predictive accuracy of the machine learning model was assessed by the area under the receiver operator characteristic curve (AUC of the ROC). The combination of feature selector and classifier with the highest AUC was used to construct the optimal radiomics model. An MRI radiomic score (Rad-score) was constructed based on the final model.

This study evaluated the performance of the predictive model from three different aspects: (1) AUC of ROC was used to reflect the predictive accuracy or discrimination power of the model. Delong test was used to compare the AUC of distinct ROCs; (2) Calibration curves were used to assess the consistency (goodness of fit) between the predictive and actual probability of poor intracranial response (SD/PD). (3) Decision curve analysis (DCA) was used to visualize the clinical utility of predictive models by quantifying the net benefit at different threshold probabilities. iPFS is an important long-term indicator reflecting the efficacy of intracranial treatment. To further explore whether a radiomic model can predict iPFS in patients treated with osimertinib, we used the X-tile software to determine the optimal cut-off point for the model’s prediction of iPFS.

SPSS 25.0 software and R software (version: 4.0.5) were used for statistical analysis. All statistical tests were two-sided. In the stepwise logistic regression analysis, a p < 0.1 was used as the standard for variable exclusion. A p < 0.05 was considered statistically significant in all the other tests.

Among the 60 EGFR-T790M-positive NSCLC patients who received second-line osimertinib therapy, 40 (66.7%) had multiple brain metastases. Totally, 115 brain metastases were included in the analysis. We randomly split the 115 brain metastases into the training (N = 80) and the validation (N = 35) cohort with a ratio of 7:3 (Table 1; Supplementary Table S1). To reduce bias due to cluster correlation, brain metastases from the same patient were assigned to the same group, as recommended by previous studies (24, 36). The baseline characteristics of the two groups were well balanced.

The overall intracranial efficacy of the second-line osimertinib is shown in Supplementary Figure S1. In aggregated, 9 (7.8%), 72 (62.6%), 30 (26.1%), and 4 (3.5%) brain metastases achieved the best therapeutic response of CR, PR, SD, and PD, respectively. The objective response rate (CR + PR) of brain metastases was 81/115 (70.4%). At the end of follow-up, the median iPFS was 15.4 months (95% CI: 14.4–23.9 months) for the total cases.

Univariate logistic regression was performed to explore the predictive value of clinical factors and MRI morphological features on the intracranial efficacy of the second-line osimertinib (Table 2A). Among clinical factors, only age > 60 years (OR: 4.06, 95%CI: 1.64–10.05, p = 0.002) was associated with an unfavorable intracranial response (SD/PD). In MRI morphological features, ring enhancement (OR: 2.53, 95%CI: 1.12–5.75, p = 0.026) and peritumoral edema (OR: 4.72, 95%CI: 1.95–11.45, p = 0.001) predicted high risk of SD/PD. Other parameters were not related to the intracranial efficacy of osimertinib. Multivariate analysis further revealed that age, ring enhancement, and peritumoral edema were independent prognosticators (Table 2B). The AUC of the predictive model established based on these three factors was 0.773 for the total patients.

We then investigated the predictive significance of MRI radiomic features in predicting the intracranial efficacy of the second-line osimertinib. After preprocessing of MRI images, 1,223 T1CE MRI radiomic features were extracted. These radiomic features were subsequently standardized by Combat harmonization and Z-score transformation. After ICC analysis, 1,191 (97.4%) features with ICC > 0.9 were retained for further analysis (Figure 2A). Univariate analysis indicated that 192 radiomic features were predictors of intracranial efficacy of the second-line osimertinib (Figure 2B), including 109 (56.8%) risk factors and 83 (43.2%) protective factors. We then applied the findCorrelation function in R package “caret” to filter out redundant radiomic features (Figure 2C). Initially, 25.65 and 16.67% of radiomic features shared a correlation coefficient (R) > 0.7 and R > 0.8, respectively, which dropped to 6.7 and 3.1% after we removed the high-correlated features. Eventually, 42 MRI radiomic features remained and subsequently subjected to the modeling analysis.

We then used the data of the training cohort to develop the machine learning model for predicting the intracranial efficacy of the second-line osimertinib. During this, we built a total of 64 machine learning models based on eight feature selectors and eight classifier algorithms. Ultimately, the model constructed using the mRMR feature selector and the stepwise logistic regression classifier achieved the highest average AUC in the 5-fold cross-validation, thereby establishing it as the final modeling approach (Supplementary Figure S2). The highest AUC from a single fold in the 5-fold cross-validation were 0.879 for the training cohort and 0.786 for the validation cohort. Consequently, this model was confirmed as the final model (Figure 3A). A nomogram was used to visualize the predictive model (Figure 3B). Each radiomic feature is assigned a score on its respective scale in the nomogram. To use it, find the patient’s value for each radiomic feature and draw a vertical line to the “Score” scale to get the individual score. Add up the scores for all features to obtain the “Total score.” Finally, use the “Total score” to draw a vertical line down to the “SD/PD Risk” scale to determine the probability of SD/PD.

Multivariate logistic regression analysis indicated that after the inclusion of the MRI radiomic model, clinical factors and MRI morphological features were no longer predictors of intracranial efficacy of the second-line osimertinib. In contrast, MRI radiomic model remained a robust independent predictor (Supplementary Table S2).

We further validated the MRI radiomic model in different aspects (Figures 3C–K). The AUCs of the ROC curves of the MRI radiomic model were 0.879, 0.786, and 0.832 in the training, validation, and total cohort, respectively, which were remarkably higher than those of the clinical-MRI morphological model (Figures 3C,F,I). In the validation cohort, the model had 84.0% specificity and 73.5% sensitivity at the optimal threshold value. Besides, decision curve analysis showed that the MRI radiomic model had a higher net benefit than the clinical-MRI morphological model at various threshold probabilities, revealing the good clinical utility of the MRI radiomic model (Figures 3D,G,J). Furthermore, calibration curves also revealed satisfactory agreement between predictive and observational probabilities of poor intracranial efficacy (SD/PD) in the second-line osimertinib treatment (Figures 3E,H,K). However, incorporating clinical factors and MRI morphological features into the MRI radiomic model leaded to few improvements of the MRI radiomic model.

As shown in Figure 4A, the MRI radiomic model could clearly distinguish NSCLC patients with favorable and poor intracranial efficacy in the second-line osimertinib treatment. Patients with a higher MRI radiomic score (Rad-score) are more prone to have a poor intracranial response (SD/PD). Brain MRI images of NSCLC patients with distinct Rad-score who achieved different intracranial response to osimertinib are shown in Supplementary Figure S3.

We further explored the relationship between MRI Rad-score and patients’ iPFS in the second-line osimertinib treatment. The X-tile analysis suggested that the optimal MRI Rad-score cutoff value for predicting iPFS was 199. The median iPFS of patients with MRI Rad-score > 199 and ≤ 199 were 3.0-Mo and 15.4-Mo, respectively (p < 0.001) (Figure 4B).