Deep learning analysis of MRI to assess rectal cancer treatment - Selby, Heather M.; Son, Ashley Y.; Sheth, Vipul R.; Wagner, Todd H.; Pollom, Erqi L.; Morris, Arden M. (Frontiers in Oncology, 2026)
IntroductionTraditional neoadjuvant therapy for locally advanced rectal cancer (LARC) results in pathologic complete response (pCR) in approximately 15% of patients, supporting non-operative strategies for those with clinical complete response (cCR). The subjectivity and variability in MRI-based cCR assessments highlight the need for objective, quantitative tools.ObjectiveTo develop deep learning models for automated rectal tumor segmentation on pre- and post-treatment MRIs, and to identify radiomic features differentiating cCR from non-cCR patients.Materials and methodsWe retrospectively analyzed pre- and post-treatment MRIs from 37 LARC patients enrolled in a Phase 2 TNT trial (NCT04380337). Rectal tumors were segmented on T2-weighted images by two data scientists, refined by a radiologist (reference standard), and independently segmented by a fellow. For pre-treatment segmentation, Model 1 (baseline; n=37) was trained on reference cases, then used to generate pseudo-labels for 81 additional cases. Model 2 (semi-supervised; n=118) was trained on the combined dataset. Model 3 (baseline; n=37) was trained on post-treatment cases. Radiomic features were extracted from post-treatment ADC maps, filtered by reproducibility (ICC ≥0.8) and redundancy (Spearman ρ≤0.95), then analyzed using unsupervised hierarchical clustering.ResultsFor pre-treatment segmentation, radiologist-fellow inter-rater agreement was DSC =0.748±0.092. Model 1 achieved mean DSC =0.682±0.254 versus the radiologist, significantly lower than inter-rater agreement. Model 2 improved performance to mean DSC =0.769±0.214 (mean gain =0.087; 12.8% relative improvement; p<0.001), slightly outperforming inter-rater agreement. For post-treatment segmentation, inter-rater agreement declined to mean DSC =0.362±0.256, while Model 3 achieved mean DSC =0.175±0.231 versus the radiologist, reflecting challenges from treatment-induced tissue changes affecting both automated models and human raters. Radiomic clustering revealed two distinct patient groups aligned with cCR and non-cCR status.ConclusionThis study demonstrates the feasibility of deep learning-based automated segmentation and radiomic profiling for differentiating treatment response in rectal cancer. Semi-supervised learning with pseudo-labeled data significantly improved segmentation performance, offering a practical approach to overcome limited annotations. Radiomic features warrant validation in larger multi-center studies for clinical translation.