The integration of molecular information into medical imaging has emerged as a key direction for improving drug sensitivity prediction in cancer treatment. Traditional imaging methods primarily focus on visible tumor features, such as size, shape, and contrast enhancement patterns. While clinically useful, these visual cues often fail to reflect the molecular diversity that underlies variations in treatment response (10). Radiogenomics has established a foundational link between imaging features and molecular characteristics, enabling researchers to identify image-based biomarkers that correspond to specific biological pathways (11). In recent years, deep learning—particularly convolutional neural networks (CNNs)—has been increasingly used to automate this process. These models can learn complex patterns in imaging data that correlate with molecular traits, such as those linked to drug resistance (12). Public datasets like The Cancer Genome Atlas (TCGA) and The Cancer Imaging Archive (TCIA) have supported the development of such integrated models. Studies have shown that incorporating gene expression data into imaging pipelines can significantly improve predictive accuracy (13). Moreover, integration has expanded beyond transcriptomics to include other molecular dimensions such as somatic mutations, copy number variations, and DNA methylation profiles, enriching image-based classification with multi-omics information (14). Despite this progress, several challenges remain. One major issue is the variability across datasets—in both imaging modalities and molecular profiling platforms—which complicates model training and generalization. Addressing this requires advanced normalization methods and domain adaptation strategies (15). Another ongoing research focus is model interpretability. It is critical to ensure that the image features used for prediction correspond to meaningful biological phenomena rather than artifacts or correlations without causation (16). Improving these aspects is essential to gain clinical trust and to enable the practical deployment of molecular-informed imaging models in precision oncology (17).

Deep learning models have emerged as essential tools for predicting drug response due to their capability to capture high-dimensional and non-linear relationships intrinsic to biomedical data (18). Traditional approaches to drug sensitivity prediction in cancer, which have predominantly utilized cell line assays or patient-derived xenografts, are constrained by substantial resource requirements and limited scalability (19). The availability of extensive pharmacogenomic datasets, including the Genomics of Drug Sensitivity in Cancer (GDSC) and the Cancer Cell Line Encyclopedia (CCLE), has enabled the development of deep learning architectures that map comprehensive molecular profiles to therapeutic outcomes (20). Autoencoders, graph neural networks, and multimodal deep learning frameworks have been deployed to integrate genomic, transcriptomic, and proteomic data with drug molecular characteristics to forecast efficacy (21). Incorporation of imaging modalities into predictive models has facilitated the learning of joint feature representations that simultaneously capture phenotypic traits and molecular determinants of drug response (22). Multimodal variational autoencoders (MVAEs) have been employed to simultaneously encode histopathology images and molecular profiles, resulting in enhanced prediction performance across heterogeneous cancer types (23). Optimization of these models frequently involves specialized loss functions, including contrastive loss and triplet loss, to ensure alignment between multimodal feature spaces and therapeutic responses (24). Despite advances, significant obstacles persist, notably the scarcity of labeled data, pronounced heterogeneity across cancer subtypes, and difficulties in achieving model generalization across diverse patient cohorts (25). Strategies such as transfer learning and few-shot learning are being actively explored to address these limitations, promoting the development of robust, scalable, and clinically translatable deep learning systems for drug response prediction (26).

The integration of multimodal data, encompassing imaging, molecular profiles, clinical metadata, and therapeutic outcomes, represents a pivotal approach for advancing drug sensitivity prediction in oncology (27). Multimodal data fusion techniques are typically categorized into early fusion, intermediate fusion, and late fusion, with each strategy offering distinct trade-offs regarding information preservation and model complexity (28). Early fusion methods involve the concatenation of raw features from disparate modalities prior to modeling, although they often encounter challenges associated with dimensionality explosion and modality dominance (29). Intermediate fusion approaches, which entail learning-modality-specific latent representations before their integration through mechanisms such as attention or latent alignment, have demonstrated a superior balance between modality fidelity and cross-modal interaction (30). Late fusion techniques independently model each modality and subsequently amalgamate predictions using ensemble strategies, enhancing robustness at the potential cost of synergistic feature utilization (31). The application of transformer architectures for multimodal fusion, leveraging cross-attention mechanisms to dynamically model inter-modal dependencies, has recently achieved notable success in enhancing predictive accuracy and interpretability (32). Cross-modal contrastive learning has further strengthened the ability of models to align heterogeneous modality representations within a unified embedding space, promoting generalization across diverse datasets (33). Concurrently, the incorporation of explainable artificial intelligence (XAI) techniques into multimodal fusion frameworks has facilitated the attribution of predictive outcomes to specific modalities, fostering transparency and clinical confidence in model outputs. As multimodal fusion methodologies continue to evolve, they offer transformative potential for personalizing cancer therapy through comprehensive and molecularly informed image-based drug response prediction.

This section systematically introduces the core components of the proposed framework for imaging problems. Section 3.2 establishes the fundamental concepts and formal notations, where the imaging model is defined, key mathematical abstractions are articulated, and the problem setting is formalized. The imaging formation process is modeled as y=H(x)+n, where x denotes the latent clean image, y represents the observed degraded image, H is the degradation operator, and n denotes additive noise. The degradation operator H can encapsulate blurring, downsampling, or a mixture of complex distortions. This foundation ensures that subsequent developments are rooted in precise mathematical formulations and notational consistency. Section 3.3 presents the newly proposed imaging model, which addresses the complexities inherent in real-world visual degradations. Instead of adopting rigid assumptions on noise distribution or blur kernels, the model parameterizes the degradation process using adaptable structure that adapts to spatially varying conditions. Let the degradation operator be parameterized as H_θ, where θ represents learnable parameters inferred from the degraded observation y. A spatially adaptive convolutional mechanism is embedded within H_θ to model heterogeneous distortions across the image domain. The noise n is treated as a realization from a location-dependent distribution n∼N(0,σ2(x)), where σ2(x) varies with the underlying content. This formulation significantly enhances the model’s capacity to handle diverse degradation types and non-stationary noise patterns.

Following the model construction, Section 3.4 proposes a progressive optimization strategy for model training. Departing from conventional single-pass or heuristic-guided approaches, the optimization unfolds over multiple refinement stages. At each stage t, the reconstruction x^t is updated by selectively activating high-confidence regions, measured by a certainty map C(x^t). The certainty map is derived from the posterior distribution over the latent image space and guides the learning objective to prioritize reliable regions before addressing more uncertain parts. Formally, the update rule incorporates a masked loss function Lt=∥C(x^t)⊙(Hθ(x^t)−y)∥2, where ⊙ denotes element-wise multiplication. This progressive mechanism not only stabilizes convergence but also mitigates the risk of overfitting to corrupted regions during early stages. Throughout these sections, mathematical rigor and methodological clarity are emphasized. Operators, distributions, and functions are carefully defined to ensure transparent interpretations. Optimization objectives are designed to be both theoretically sound and computationally tractable. The methodology combines classical principles from inverse problems with recent advances in deep learning, resulting in a hybrid framework that leverages domain-specific priors while maintaining flexibility through data-driven learning. The degradation operator Hθ, certainty map C(x^t), and noise modeling σ²(x) are seamlessly integrated into a unified architecture, allowing coherent end-to-end training. The overall approach decomposes the imaging reconstruction problem into three structured modules: degradation modeling, noise-adaptive regularization, and confidence-driven optimization. The degradation modeling module parameterizes distortions through a learnable convolutional structure, the noise-adaptive regularization module captures spatially varying uncertainty, and the confidence-driven optimization module progressively refines the reconstruction by exploiting spatial reliability. This decomposition not only improves interpretability but also yields significant empirical performance gains across multiple tasks.

The proposed method establishes a robust framework for a wide range of imaging tasks, including deblurring, denoising, and super-resolution. Extensive experimental results demonstrate that the method consistently surpasses state-of-the-art baselines across various benchmarks. The adaptability to non-uniform degradations and the progressive learning paradigm lead to improved reconstruction fidelity and robustness under both synthetic and real-world degradation scenarios. Each component introduced in the following sections is meticulously designed to interoperate, resulting in a coherent system that balances modeling accuracy, computational efficiency, and learning stability. Section 3.2 specifies the imaging problem setup and defines the notation used throughout the paper. Section 3.3 elaborates the detailed architecture and parameterization of the proposed degradation model. Section 3.4 describes the progressive optimization procedure, detailing how certainty maps are constructed and utilized to guide learning. This section provides a detailed exposition of the methodology, preparing the foundation for the theoretical analysis and experimental validation presented in subsequent parts. By carefully integrating model design, adaptive regularization, and progressive optimization, the proposed framework achieves superior performance with enhanced theoretical guarantees and practical effectiveness.

To provide a clearer understanding of the overall architecture, a system-level overview of the proposed framework is illustrated in Figure 1. The model consists of two main components: DSINet, which performs spatially adaptive dynamic filtering and multi-resolution refinement, and PSGO, which carries out progressive structure-guided optimization. The entire pipeline begins with degradation modeling and noise-adaptive regularization, followed by a dual-stage reconstruction strategy that combines content-awareness and confidence-driven refinement. Figure 2 depicts the internal structure of DSINet. It dynamically generates spatially adaptive filters via a vision transformer and text-guided token sampler. These filters are applied in a context-aware manner to the input image. A structure-preserving attention mechanism ensures that biologically important regions are retained. The model further integrates multi-resolution refinement and uncertainty modeling to support robust image representation. Figure 3 shows the three stages of PSGO, namely: adaptive confidence-guided decomposition, where reliable and uncertain regions are separated using a learned confidence map; iterative restructuring with uncertainty adaptation, which updates graph structures using attention-based mechanisms; and dynamic graph-regularized propagation, which propagates information across spatially consistent regions to obtain a high-fidelity reconstruction. These components together enable the model to adaptively handle noise, structural variation, and uncertainty in biomedical images.

This section formalizes the imaging problem considered in this work by introducing the mathematical notations, degradation models, and key assumptions that underpin subsequent developments.

Let x ∈ ℝn denote the latent sharp image, and y ∈ ℝn the observed degraded image. The degradation process is modeled as (Equation 1):

where H:ℝn→ℝn is a degradation operator and є ∈ ℝn represents additive noise.

A standard instance of H(·) involves convolution with a blur kernel k ∈ ℝm (Equation 2):

where ∗ denotes the convolution operation under specified boundary conditions.

For spatially variant degradations, the degradation operator is expressed as (Equation 3):

where Ω(i) defines the neighborhood around pixel i, and k_i,jare location-dependent kernel weights. The noise term $ accounts for both Gaussian and structured perturbations decomposed as (Equation 4):

where єG represents Gaussian noise and єS models sparse outliers such as impulsive noise or saturation 217 artifacts.

s, let H∈ℝn×n denote the convolution matrix associated with k, and E∈ℝn×n the covariance matrix of the noise. The degradation model becomes (Equation 5):

where e∼N(0,E) in the Gaussian component.

The ill-posed nature of the inverse problem necessitates regularization. One widely adopted prior assumes sparsity in the gradient domain (Equation 6):

where ∇x=(Dhx,Dvx), and Dh, Dv are horizontal and vertical difference operators, respectively.

To capture high-level structures, a feature extraction operator F:ℝn→ℝp is introduced (Equation 7):

where z∈ℝp encodes salient features including edges, textures, or semantic patterns.

Modeling complex degradations often requires incorporating nonlinearities. The forward model is thus extended to (Equation 8):

where G:ℝn→ℝn denotes a nonlinear transformation accounting for effects such as clipping, gamma correction, or sensor-specific distortions.

The imaging recovery objective is formulated as an optimization problem (Equation 9):

where D(·,·) measures the discrepancy between the observation and reconstruction.

In blind deconvolution settings, both the latent image x and the blur kernel k are unknown (Equation 10):

To explicitly handle structured noise, an auxiliary variable o∈ℝn is introduced, leading to the modified observation model (Equation 11):

and the corresponding joint estimation problem is (Equation 12):

where Ψ(·) is a sparsity-promoting regularizer and λ is a positive parameter balancing fidelity and noise modeling.

To stabilize the inversion when H is ill-conditioned, a Tikhonov regularization is introduced (Equation 13):

where L denotes a Laplacian or higher-order differential operator.

Multiscale approaches are utilized to progressively refine reconstructions. Let {xs}s=1S represent the latent images at multiple scales, with the degradation model at each scale formulated as (Equation 14):

where Hs(·) and єs denote the degradation operator and noise at scale s, respectively.

An attention mechanism is introduced to adaptively weigh spatial locations. Defining an attention map α:ℝn→[0,1]n, the degradation model becomes (Equation 15):

where ⊙ denotes element-wise multiplication and η models dominant noise.

All subsequent methods are built upon this general formalism, enabling flexible modeling of diverse degradation processes and guiding the design of robust imaging recovery algorithms.

In this section, we present the proposed dynamic structure-aware imaging network (DSINet), a novel imaging framework specifically tailored to robustly reconstruct high-fidelity latent images from heavily degraded observations. Traditional imaging models often fail under severe degradations due to their reliance on static filters and inadequate modeling of structural information. DSINet fundamentally rethinks this paradigm by introducing three key innovations that jointly enable dynamic, structure-aware, and uncertainty-guided imaging. The detailed design and mathematical formulation of DSINet are provided below. The DSINet architecture incorporates multimodal interaction by integrating visual and textual tokens within the adaptive filtering module. As illustrated in Figure 2, both Vision Transformer embeddings and language-derived query embeddings are passed to a token-level sampler. The resulting informative tokens serve as semantic priors to guide the generation of spatially adaptive filters in A_θ(x). These tokens influence the selection and modulation of convolutional kernels in a data-dependent manner. The framework enables conditioning of local image processing on both morphological features and semantic textual cues, enhancing both accuracy and interpretability. The structure-preserving module further fuses adaptive outputs with anatomical priors to retain spatial fidelity. To improve consistency between the mathematical formulation and architectural illustration, a mapping table is provided (see Table 1) to clarify the correspondence between pixel-level variables and token-level representations. This alignment facilitates a better understanding of how vision and text modalities are integrated within DSINet. Pixel-wise operators such as A_θ(x) and K(f_i) are dynamically modulated by token embeddings derived from both the Vision Transformer and Text-Guided Sampler. This joint representation supports spatially adaptive processing that remains semantically grounded in both visual and linguistic contexts.

In this section, we introduce progressive structure-guided optimization (PSGO), a novel iterative framework designed to refine the imaging reconstruction produced by DSINet. The core idea of PSGO is to progressively focus on reliable structural components while dynamically adjusting the optimization pathway based on intermediate recovery states (as shown in Figure 3).

The BraTS Dataset (34) is a widely used benchmark in the field of medical image analysis, specifically designed for the segmentation of brain tumors in multi-modal magnetic resonance imaging (MRI) scans. It contains annotated images from patients with gliomas, both high-grade and low-grade, across multiple institutions and scanners, ensuring diversity and robustness. The dataset includes several MRI sequences such as T1, T1-contrast enhanced, T2, and FLAIR, providing comprehensive structural information crucial for tumor delineation. The ground truth labels are manually segmented by experienced radiologists, distinguishing tumor core, enhancing tumor, and edema regions. BraTS has been the focus of multiple international challenges, driving advancements in automated brain tumor segmentation and serving as a reference for validating new methodologies. Its rigorous curation, multi-center acquisition, and detailed annotations make it indispensable for developing and benchmarking machine learning algorithms for neuro-on. The OASIS Dataset (35) is an open-access collection of brain imaging data aimed at advancing the understanding of normal aging and Alzheimer’s disease. It includes cross-sectional and longitudinal MRI scans from a large cohort of participants ranging in age and cognitive status, from young healthy adults to elderly individuals with varying degrees of cognitive impairment and dementia. The dataset provides structural MRI volumes, demographic information, and clinical assessments such as the Clinical Dementia Rating (CDR). OASIS is widely used for studying brain morphometry, the progression of neurodegenerative diseases, and for training and validating automated brain segmentation and classification algorithms. The careful design and broad scope of OASIS enable comprehensive studies on aging, neuroanatomical changes, and disease progression, fostering reproducibility and comparison across different research groups and methodologies in the neuroscience community. The LUNA16 Dataset (36) is a large-scale, curated resource for the detection of lung nodules in computed tomography (CT) images. Derived from the publicly available LIDC-IDRI database, LUNA16 consists of a carefully selected subset of CT scans with annotated pulmonary nodules by multiple radiologists. Each nodule annotation is provided with spatial coordinates and diameter, ensuring precise localization and quantification. The dataset focuses on nodules greater than 3 mm in diameter, reflecting clinical relevance in lung cancer screening. LUNA16 has served as the foundation for the Lung Nodule Analysis Grand Challenge, enabling standardized evaluation and comparison of algorithms for automated nodule detection and classification. Its comprehensive annotation protocol, high-resolution CT data, and public accessibility have made it a benchmark for advancing computer-aided detection systems in thoracic imaging. The MURA Dataset (37) is a large-scale musculoskeletal radiograph dataset designed for the development and assessment of algorithms for abnormality detection in bone X-rays. It contains over 40,000 images from more than 14,000 studies, spanning seven standard upper extremity radiographic examinations such as the elbow, finger, hand, humerus, forearm, shoulder, and wrist. Each study is labeled by board-certified radiologists as either normal or abnormal, reflecting clinically relevant findings encountered in routine practice. The MURA dataset enables both classification and localization tasks, as it provides image-level labels and, for a subset, bounding box annotations. Its scale, diversity of anatomical regions, and expert labeling have made it a reference dataset for evaluating deep learning models in musculoskeletal imaging and for facilitating research on automated abnormality detection in radiographs. A detailed summary of all datasets used in this study is provided in Table 2, including the disease type, imaging modality, availability of molecular information, classification endpoints, sample size, and the data splitting strategy. This table is intended to clarify the experimental settings and facilitate reproducibility and comparison with related studies.

All experiments were conducted on a workstation equipped with NVIDIA RTX 3090 GPUs, 256GB RAM, and Intel Xeon CPUs, running Ubuntu 20.04 LTS. The full framework was implemented using PyTorch (version 1.12) with CUDA 11.3 acceleration, and random seeds were fixed at the beginning of each run to ensure reproducibility. The preprocessing steps included intensity normalization, resampling to a standardized spatial resolution (voxel spacing for 3D datasets or pixel spacing for 2D images), and cropping or padding to fixed input dimensions. For MRI datasets, additional neuroimaging-specific steps such as bias field correction and skull stripping were applied to remove non-brain tissue. Data augmentation strategies were applied during training, including random rotations, scaling, horizontal and vertical flipping, elastic deformations, and intensity jittering, to enhance robustness and reduce overfitting. In all experiments, the DSINet framework served as the primary backbone for both image enhancement and downstream medical image analysis. For volumetric 3D datasets such as BraTS and LUNA16, a 3D variant of DSINet was constructed using 3D convolutional layers and patch-based input processing. For 2D datasets such as MURA and OASIS, a 2D version of DSINet was used, in which a Vision Transformer extracted visual tokens that were then fused with text- or molecule-derived semantic embeddings through a cross-attention module. The progressive structure-guided optimization (PSGO) module was integrated across all configurations to refine predictions by iteratively updating confidence-aware spatial features. Model weights were initialized using He initialization for convolutional layers and Xavier initialization for fully connected layers. Training was conducted with a batch size of 4 for 3D datasets and 32 for 2D datasets, using the Adam optimizer with an initial learning rate of 1 × 10⁻⁴. The learning rate was reduced by a factor of 0.5 when the validation loss did not improve for 10 consecutive epochs. Early stopping with a patience of 20 epochs was applied to prevent overfitting, and training proceeded for a maximum of 200 epochs. The final model used for evaluation corresponded to the checkpoint with the lowest validation loss. Although DSINet is initially designed as a structure-aware image enhancement model, the learned representations from its encoder were reused for classification and segmentation tasks. For classification, a softmax layer was added after global average pooling on the encoder outputs. For segmentation, a lightweight decoder was attached to the multi-scale visual features, trained end-to-end using a combination of Dice loss and binary cross-entropy loss. For classification tasks, categorical cross-entropy was used. Evaluation was performed on held-out test sets using metrics appropriate to each task, including Dice similarity coefficient (DSC), intersection over union (IoU), sensitivity, specificity, area under the ROC curve (AUC), accuracy, and average precision. Each experiment was repeated three times with different random seeds, and mean and standard deviation were reported to ensure statistical robustness. Data splits were performed in a stratified manner to maintain class balance. External validation was also conducted when possible, using either publicly available test sets or cross-institutional data, to assess the generalizability of the proposed method.

To specify the molecular data utilized in this study, gene expression profiles were obtained from The Cancer Genome Atlas (TCGA) and the METABRIC dataset. In particular, the TCGA-LGG (low grade glioma) and TCGA-BRCA (breast cancer) cohorts provided matched histopathological images and transcriptomic data. The expression values were normalized, filtered using variance thresholds, and z-score standardized. Sample correspondence across modalities was ensured using unique patient identifiers. During model construction, molecular features were concatenated with visual embeddings to form a unified representation that incorporates both anatomical and biological information. The multimodal learning architecture adopts two parallel encoding pathways. Visual information is processed through a Vision Transformer (ViT) to extract semantic features from imaging data. In parallel, molecular vectors are processed using a lightweight transformer encoder. The resulting feature embeddings are fused via a cross-attention module, which dynamically aligns and weights information from both modalities. This design allows molecular context to influence spatial filtering and prediction, improving robustness under ambiguous or noisy visual conditions. The joint representation is subsequently refined through structure-aware filtering and confidence-guided optimization mechanisms implemented in DSINet and PSGO.

Tables 3 and 4 illustrate a systematic contrast between our algorithm and high-performing baseline methods across four commonly adopted datasets: BraTS, OASIS, LUNA16, and MURA. For each dataset, multiple standard evaluation metrics are reported, including accuracy, precision, recall, and F1 score, enabling a robust and multi-faceted assessment of classification and detection performance. On the BraTS dataset, which focuses on brain tumor segmentation, our method achieves a significant improvement over established models such as ResNet50, DenseNet121, EfficientNet-B0, ViT, ConvNeXt, and DeiT. Specifically, our model obtains an accuracy rate of 92.61 ± 0.03, which is 2.39% higher than ConvNeXt, the best-performing baseline. Similarly, on the OASIS dataset, which addresses Alzheimer’s disease prediction from structural MRI, our method attains an accuracy rate of 91.98 ± 0.02, surpassing the closest SOTA competitor, ConvNeXt, by nearly 3%. In addition to overall accuracy, our model consistently outperforms competitors across all other key metrics, demonstrating balanced improvements in both precision and recall, leading to higher F1 scores and thus more reliable detection. The standard deviations across repeated experiments are also notably lower for our approach, underscoring its robustness and reproducibility. Performance gains are not limited to classification; improvements are observed in segmentation precision, indicating that our method can better delineate boundaries in complex neuroimaging data, which is critical for clinical applicability. In the case of the LUNA16 and MURA datasets, which focus on lung nodule detection in chest CT and musculoskeletal abnormality detection in radiographs, respectively, the superiority of our approach remains evident. On LUNA16, our method records an accuracy of 91.48 ± 0.03, outperforming ConvNeXt-Tiny and Swin-Transformer by margins of nearly 3%. For the MURA dataset, which is particularly challenging due to high inter-class variability and subtle abnormality cues, our model achieves an accuracy rate of 87.76 ± 0.03, distinctly higher than all baseline models, with a significant boost in both recall and F1 score. This indicates that our method not only correctly identifies more abnormal cases but also maintains a low rate of false positives and negatives, which is vital in clinical screening settings. Our analysis reveals that while transformer-based methods such as ViT, Swin-Transformer, and MAE have demonstrated notable improvements over traditional convolutional neural network (CNN) architectures, especially in capturing global context and long-range dependencies, their performance still lags behind our proposed approach. One likely reason is that our model introduces adaptive feature fusion modules and attention mechanisms specifically designed for medical imaging, allowing for enhanced extraction of domain-relevant features and more effective integration of multi-scale contextual information. This design mitigates some of the common pitfalls encountered by standard transformers, such as overfitting on small datasets and insufficient representation of fine-grained patterns.

The marked improvement of our method across diverse datasets and imaging modalities can be attributed to several core design principles and technical innovations. First, our method leverages advanced regularization strategies and cross-domain data augmentation, which collectively enhance generalization and mitigate dataset-specific biases. For instance, the inclusion of modality-adaptive normalization enables the model to dynamically adjust to intensity and distributional variations present in MR, CT, and X-ray images, addressing a key limitation of many existing models. Second, the hierarchical attention mechanism integrated into our network architecture enables both global and local context modeling, ensuring that the model can focus on subtle anatomical abnormalities while also leveraging broader contextual cues, a capability essential for accurate medical image interpretation. Third, as highlighted in method ablation studies and reflected in Table 3, our approach incorporates task-specific loss functions and balanced sampling strategies, effectively addressing the class imbalance issues prevalent in medical data sets. Furthermore, efficient parameterization of our network, with a focus on lightweight modules and reduced computational overhead, allows practical deployment in real-world clinical scenarios without sacrificing performance. This efficiency is further evidenced by the relatively low variance in performance metrics across repeated runs, demonstrating stability and reliability.

To contextualize the performance comparisons, a brief summary of the baseline models is presented. ResNet50 and ResNet34 are classical convolutional neural network (CNN) architectures that rely on residual connections to facilitate gradient flow and improve training stability. They are widely used in medical imaging tasks but are generally limited in modeling long-range dependencies and require large datasets to generalize effectively. DenseNet121 and DenseNet169 improve upon standard CNNs by introducing dense connectivity between layers, promoting feature reuse and better gradient propagation. These models are efficient and parameter-light but still rely on fixed receptive fields, which may be suboptimal for highly variable biomedical structures. EfficientNet-B0 and MobileNetV2 are lightweight CNN variants optimized for performance-efficiency trade-offs. While effective in resource-constrained environments, their relatively shallow architectures may limit expressiveness in complex classification tasks. Vision Transformers (ViT), Swin-Transformer, and MAE belong to the transformer-based family of models. ViT applies self-attention mechanisms directly to flattened image patches, enabling global context modeling. Swin-Transformer introduces hierarchical feature representations with shifted windows to balance local and global feature extraction. MAE leverages masked image modeling as a self-supervised pretraining method. These models offer improved representation learning but often require large-scale pretraining and are sensitive to data scarcity. ConvNeXt and ConvNeXt-Tiny represent a hybrid design that modernizes CNNs using architectural concepts from transformers, such as layer normalization and GELU activations, while retaining convolutional inductive biases. They offer strong performance in various vision tasks but still lack mechanisms to incorporate uncertainty or spatial confidence. In contrast, the proposed DSINet + PSGO architecture incorporates spatially adaptive filtering, structure-preserving attention, and uncertainty-aware optimization. This combination allows it to dynamically adjust to image content, selectively focus on reliable regions, and robustly integrate molecular and imaging data—capabilities that are not present in the baseline models.

To validate the contribution of each proposed component, we conducted an ablation study across the BraTS, OASIS, LUNA16, and MURA datasets. We systematically removed each key module—spatially adaptive dynamic filtering, structure-preserving attention modulation, and multi-resolution refinement with uncertainty modeling—from the full DSINet framework to examine their individual impact. The experimental results, summarized in Tables 5 and 6, demonstrate consistent performance degradation across all major evaluation metrics, comprising performance measures—accuracy, precision, recall, and F1—under conditions of single-module ablation.

The exclusion of spatially adaptive dynamic filtering leads to the most substantial accuracy and recall drops, reflecting its role in dynamically adjusting filtering behavior according to local image degradations. Removing structure-preserving attention modulation notably reduces precision and F1 scores, indicating its effectiveness in preserving essential structural information during reconstruction. The omission of multi-resolution refinement and uncertainty modeling results in significant decreases in recall and F1 scores, underscoring the importance of capturing multi-scale dependencies and managing prediction uncertainty. These results affirm that all three components jointly address diverse challenges in medical image reconstruction and are indispensable for achieving robust and high-fidelity results.

To enhance the interpretability of the molecular-informed classification framework and evaluate robustness under clinical conditions, several additional experiments were conducted. Each dataset was processed independently, and separate models were trained due to variations in imaging modalities and the presence or absence of molecular annotations. Molecular features were incorporated only for datasets where such data were available (like APOE genotype in the OASIS dataset). A unified model across all datasets was intentionally avoided to prevent confounding due to heterogeneous input distributions. To quantify the contribution of molecular features, a SHAP (SHapley Additive exPlanations) analysis was performed on the OASIS dataset. Table 7 presents the importance ranking of features used in the classification task. The APOE genotype was found to be the most predictive molecular factor, exceeding key imaging-derived features in impact. In a modality ablation study, molecular features were removed at test time to assess their impact on predictive performance. As shown in Table 8, a noticeable decline in F1 score was observed, particularly on datasets with informative molecular annotations, highlighting the value of incorporating such features. To simulate real-world clinical scenarios involving incomplete annotations, molecular features were randomly masked at rates of 10%, 30%, and 50%. The classification performance degraded gracefully, with less than 6% decline in F1 score even under 50% missing molecular input, indicating that the model retained robustness under partial information.

To complement the ablation studies and provide insight into computational feasibility, this section reports the efficiency metrics of the full and simplified versions of the proposed architecture. Table 9 summarizes the key performance indicators: number of trainable parameters, GPU memory consumption, training time per epoch, and inference time per image. All experiments were conducted on a single NVIDIA RTX 3090 GPU with 24GB memory. The results indicate that the full version of DSINet + PSGO requires approximately 42.8 million parameters and 15.6 GB of GPU memory during training. Inference time is 247 ms per image on average. In contrast, the lightweight version—created by reducing ViT depth and disabling multiscale uncertainty refinement—achieves a 45% reduction in latency and over 50% memory savings, with only a 2.7% absolute drop in F1 score on the BraTS dataset. These findings suggest that while the full model offers maximum accuracy, lighter configurations may be more suitable for real-time or mobile deployment scenarios. Such trade-offs between accuracy and efficiency can be selected based on deployment context. Future work will further explore model pruning and quantization techniques to reduce inference overhead in clinical applications.

To strengthen the translational relevance of the proposed method, additional experiments were conducted to explore whether the image regions highlighted by DSINet correspond to biologically meaningful pathways and molecular processes related to drug sensitivity. A subset of 116 matched cases from the TCGA-LGG cohort was used, where both histopathological whole-slide images and transcriptomic data were available. For each case, attention maps were extracted from DSINet outputs. The top 10% most salient regions were localized and spatially registered back to the original slides. Gene expression profiles from the same patients were used to calculate pathway activity scores using single-sample Gene Set Enrichment Analysis (ssGSEA), with KEGG, Reactome, and Hallmark gene sets as references. Figure-level attention saliency and patient-level transcriptomic profiles were then correlated. A two-group comparison (high-attention vs. low-attention regions) revealed that the former exhibited significantly higher activity in several known drug-response pathways. The enriched pathways include PI3K/AKT signaling, p53 pathway, mismatch repair, and DNA damage response, all of which are implicated in treatment resistance or sensitivity in glioma and other cancers. The top enriched pathways are summarized in Table 10, along with FDR-adjusted p-values computed via Benjamini–Hochberg correction. These findings provide supporting evidence that the attention-driven outputs of the model are not only spatially meaningful but also biologically interpretable, reinforcing the clinical transparency of the framework. Future work will focus on embedding these biological priors into the model architecture and further validating these findings through clinical collaborations and pathway-aware training strategies.

To assess whether the proposed framework can generalize across cancer types, a cross-disease transfer experiment was performed. DSINet + PSGO was first trained on the TCGA-LGG cohort (glioma) and then evaluated without fine-tuning on a small subset of breast cancer (BRCA) and lung adenocarcinoma (LUAD) samples obtained from TCGA, where both histopathology and molecular response proxies (pathway activation scores) were available. Table 11 presents the F1 scores of the model on each dataset under two scenarios: direct inference (zero-shot) and light fine-tuning (3-epoch transfer). The full model trained on glioma data achieved reasonable performance in both BRCA and LUAD cases, and fine-tuning further improved performance. This suggests that core architectural components such as structure-aware filtering and confidence-guided optimization are transferrable across tissue types. In addition to the experimental evidence, it is important to consider the biological basis of drug sensitivity variation across cancers. As Solimando et al. (44) note, the tumor microenvironment, immune infiltration, and bone marrow niche interactions in multiple myeloma contribute significantly to treatment resistance. Similar heterogeneity exists in breast and lung cancers, where spatial proteomics has identified localized signaling changes that influence response. These findings highlight the necessity for predictive models to either incorporate domain adaptation mechanisms or demonstrate transferability across contexts. The modular nature of DSINet + PSGO, with separate components for image filtering, structure modulation, and uncertainty-based optimization, enables future extensions involving domain adaptation, few-shot fine-tuning, or federated learning to further improve real-world applicability.

To better assess the clinical relevance of the proposed framework, additional evaluations were conducted using metrics that extend beyond conventional classification accuracy. In the context of drug sensitivity prediction, clinically meaningful assessment requires not only correct classification but also reliable probability estimates and decision-level utility. Probability calibration was first evaluated by constructing calibration curves for both the TCGA-LGG and TCGA-BRCA cohorts. The proposed DSINet + PSGO produced probability outputs that closely aligned with observed outcome frequencies, indicating strong calibration properties. In addition, the concordance index (C-index) was used to measure how well predicted probabilities ranked patients according to sensitivity likelihood. DSINet + PSGO achieved a C-index of 0.821 and 0.801 on the TCGA-LGG and TCGA-BRCA datasets, respectively, surpassing baseline models such as ConvNeXt and Swin-Transformer. To evaluate clinical decision-making utility, decision curve analysis (DCA) was performed across a range of decision thresholds. The DSINet + PSGO consistently yielded higher net benefit across clinically relevant threshold intervals (0.3–0.7), compared to default treatment strategies (like treat-all or treat-none). These results suggest that the model provides not only high predictive performance but also well-calibrated and actionable outputs for potential clinical use. A summary of the clinically relevant evaluation metrics is presented in Table 12.

To further evaluate the effectiveness of the proposed progressive structure-guided optimization (PSGO) module, a comparative experiment was conducted against several widely used traditional refinement techniques. These include Dense Conditional Random Fields (DenseCRF), mean-field inference-based smoothing, and post-hoc Gaussian filtering. All methods were applied in conjunction with DSINet, where the refinement strategy was used after the initial segmentation output from the encoder-decoder backbone.

As shown in Table 13, the PSGO module consistently outperformed traditional optimization methods across all major segmentation metrics. Compared to DenseCRF, which smooths label maps by leveraging low-level intensity similarities, PSGO utilizes confidence-aware spatial features and structure-guided refinement that adaptively modulate ambiguous regions. Furthermore, unlike fixed-rule filtering methods such as Gaussian smoothing, PSGO is task-specific and data-adaptive, optimizing both feature propagation and boundary delineation. The results demonstrate that PSGO not only preserves fine-grained anatomical boundaries but also enhances semantic coherence under challenging imaging conditions. In particular, improvements were observed in Dice score and precision, indicating reduced over-segmentation and better focus on relevant structures. This confirms the advantage of structure-aware, learnable optimization mechanisms over static or heuristic refinement techniques in the context of medical image segmentation.

Molecularly informed image-based drug response prediction refers to a modeling approach that infers drug sensitivity by integrating histopathological imaging features with transcriptomic signals. Although no direct prediction of drug response metrics such as IC50 or AUC is performed, the classification tasks conducted in this work—such as molecular subtype prediction, receptor status classification, and mutation-based grouping—serve as surrogate indicators of therapeutic response. For example, the use of ER/HER2 status in the TCGA-BRCA cohort and IDH mutation status in TCGA-LGG is clinically linked to specific treatment outcomes. The proposed framework enhances prediction accuracy by embedding molecular signals into the imaging feature extraction process. Visual and molecular features are fused using a cross-attention mechanism, and the resulting representation undergoes structure-aware filtering and confidence-based optimization. The downstream analyses, including SHAP-based interpretation and pathway enrichment, further indicate that the model captures biologically relevant drug-response pathways. While explicit drug response prediction is not conducted, the framework supports indirect inference through biologically stratified image phenotypes.