Deep learning radiomics based on multimodal imaging for distinguishing benign and malignant breast tumours - Lu, Guoxiu; Tian, Ronghui; Yang, Wei; Liu, Ruibo; Liu, Dongmei; Xiang, Zijie; Zhang, Guoxu (Frontiers in Medicine, 2024)
Objectives: To develop a deep learning radiomics model using multimodal imaging to differentiate benign and malignant breast tumours.Methods: Multimodality imaging data, including ultrasonography (US), mammography (MG) and magnetic resonance imaging (MRI), from 322 patients (112 with benign breast tumours and 210 with malignant breast tumours) with histopathologically confirmed breast tumours were retrospectively collected between December 2018 and May 2023. Based on multimodal imaging, the experiment was divided into three parts: traditional radiomics, deep learning radiomics, and feature fusion. We tested the performance of seven classifiers, namely, SVM, KNN, random forest, extra trees, XGBoost, LightGBM, and LR, on different feature models. In terms of feature fusion, under the ensemble and stacking strategies, we obtained the optimal classification model for benign and malignant breast tumours.In terms of traditional radiomics, the ensemble fusion strategy achieved the highest accuracy, AUC, and specificity, with values of 0.892, 0.942 [0.886-0.996], and 0.956 [0.873-1.000], respectively. The early fusion strategy with US, MG, and MRI achieved the highest sensitivity of 0.952 [0.887-1.000]. In terms of deep learning radiomics, the stacking fusion strategy achieved the highest accuracy, AUC, and sensitivity, with values of 0.937, 0.947 [0.887-1.000], and 1.000 [0.999-1.000], respectively. The early fusion strategies of US+MRI and US+MG achieved the highest specificity of 0.954 [0.867-1.000]. In terms of feature fusion, the ensemble and stacking approaches of the late fusion strategy achieved the highest accuracy of 0.968. In addition, stacking achieved the highest AUC and specificity, which were 0.997 [0.990-1.000] and 1.000 [0.999-1.000], respectively. The traditional radiomics and depth features of US+MG+MR achieved the highest sensitivity of 1.000 [0.999-1.000] under the early fusion strategy.This study demonstrated the potential of integrating deep learning and radiomics features with multimodal images. As a single modality, MRI based on radiomics features achieved greater accuracy than US or MG. The US and MG models achieved higher accuracy with transfer learning than the single-mode or radiomics models. The traditional radiomics and depth features of US+MG+MR achieved the highest sensitivity under the early fusion strategy, showed higher diagnostic performance and provided more valuable information for differentiation between benign and malignant breast tumours.