We evaluate our method on the Chaoyang Dataset Zhu et al. (2021), a large-scale collection of whole-slide images (WSIs) retrospectively curated from routine clinical practice at Beijing Chaoyang Hospital. Representative non-overlapping patches of size 512 × 512 pixels were extracted from diagnostically relevant regions under the supervision of board-certified pathologists. The dataset comprises four clinically significant classes: Normal mucosa, Serrated lesions (including hyperplastic polyps and sessile serrated adenomas), Adenoma (tubular/tubulovillous), and Adenocarcinoma. In total, 6,160 annotated patches are included, distributed as follows: 1,816 Normal, 1,163 Serrated, 2,244 Adenocarcinoma, and 937 Adenoma. To ensure fair comparison, we adopt the consistent train and test partition: the training set contains 1,111 Normal, 842 Serrated, 1,404 Adenocarcinoma, and 664 Adenoma samples (total = 4,021); the test set includes 705 Normal, 321 Serrated, 840 Adenocarcinoma, and 273 Adenoma samples (total = 2,139). No overlap exists between training and test slides at the patient level, thereby mitigating data leakage and enabling assessment of generalization to unseen individuals. All experiments are conducted on this standardized split. Figure 1 shows some samples of the Chaoyang dataset.

The Haar wavelet transform Haar (1909) provides a computationally efficient, orthogonal multiresolution decomposition that is particularly well-suited for capturing localized intensity discontinuities—such as cell boundaries, nuclear membranes, and glandular edges—common in histopathological images. Given a 1D discrete signal s ∈ R 2 J (with J ∈ N ), the Haar transform recursively computes approximation (scaling) coefficients a j [ n ] and detail (wavelet) coefficients d j [ n ] at scale j = 0,1 , … , J − 1 via Equations 1, 2: a j n = 1 2 a j + 1 2 n + a j + 1 2 n + 1 , (1) d j n = 1 2 a j + 1 2 n − a j + 1 2 n + 1 , (2) where a J [ n ] = s [ n ] denotes the original signal, and n = 0,1 , … , 2 j − 1 . The inverse transform reconstructs s exactly from { a 0 [ 0 ] } ∪ { d j [ n ] } j = 0 J − 1 .

For 2D images X ∈ R H × W (assuming H , W are powers of two for simplicity), the 2D Haar transform applies the 1D decomposition along rows and columns successively. At each level, it yields four subbands { A , H , V , D } , where A is the low-frequency approximation (capturing coarse tissue architecture), while H , V , and D represent horizontal, vertical, and diagonal detail coefficients, respectively—encoding fine-scale texture, edge orientation, and structural irregularities. This explicit separation of spatial frequencies enables pathology-aware feature disentanglement: for instance, dysplastic nuclei often manifest as high-magnitude responses in diagonal/high-frequency bands, whereas tumor-stroma interfaces are reflected in vertical/horizontal edges. Owing to its simplicity, invertibility, and sensitivity to abrupt intensity changes, the Haar wavelet serves as an effective prior for modeling the multi-scale heterogeneity inherent in colorectal cancer histology.

We present CRC-Former, a hierarchical deep architecture designed for whole-slide image (WSI) classification in colorectal cancer pathology. As illustrated in Figure 2a, the network processes an input histopathology image X ∈ R H × W × 3 through a multi-stage pipeline that progressively extracts and refines multi-scale, frequency-aware contextual features. The pipeline begins with a Patch Embedding layer that partitions X into non-overlapping patches of size P × P , followed by linear projection to obtain initial patch embeddings. These are then processed by a sequence of four Frequency-aware Global–Local Transformer Blocks (FGT Blocks), each operating at successively coarser spatial resolutions ( H / 4 × W / 4 , H / 8 × W / 8 , H / 16 × W / 16 , H / 32 × W / 32 ). Between consecutive FGT blocks, a Patch Merging operation reduces spatial dimensionality while doubling the channel depth—enabling efficient hierarchical feature abstraction. Each FGT block integrates Haar wavelet decomposition to decompose local patch representations into multi-frequency subbands, which are then fused via hybrid attention mechanisms to capture both global context and orientation-sensitive texture cues. After the final FGT stage, the high-level, low-resolution feature map is fed into a Cross-Scale Mamba Block (CSM Block), which serves as a global aggregator. Unlike conventional transformers or CNNs, the CSM Block leverages selective state-space modeling to dynamically fuse information across all previous resolution levels—effectively integrating fine-grained textural details from early stages with coarse semantic patterns from deeper layers. This enables the model to reason about long-range tissue organization while preserving discriminative local morphological signals. Finally, a global average pooling operation is applied, followed by a fully connected Classification Head to produce the predicted class probabilities. The entire architecture is end-to-end trainable and maintains linear computational complexity with respect to sequence length, making it scalable to gigapixel WSIs without sacrificing representational power.

The Cross-Scale Mamba Block (CSM, Figure 2b) serves as the global aggregation module of CRC-Former, designed to fuse multi-resolution features extracted by the preceding four FGT stages into a unified, context-aware representation. Formally, let { F 1 , F 2 , F 3 , F 4 } denote the feature maps output by the FGT blocks at resolutions H 4 × W 4 , H 8 × W 8 , H 16 × W 16 , and H 32 × W 32 , respectively, where each F i ∈ R H i × W i × C i . To enable cross-scale interaction, all four feature maps are first downsampled via bilinear interpolation (or strided convolution) to match the coarsest spatial resolution H 32 × W 32 , yielding (Equation 3): F ̃ i = DownSample F i , i = 1,2,3,4 , (3) where F ̃ i ∈ R H 32 × W 32 × C i . These aligned representations are then concatenated along the channel dimension, which is given by Equation 4: F cat = F ̃ 1 ‖ F ̃ 2 ‖ F ̃ 3 ‖ F ̃ 4 ∈ R H 32 × W 32 × ∑ i = 1 4 C i . (4)

The concatenated tensor F cat is reshaped into a sequence z ∈ R L × D , where L = H 32 ⋅ W 32 denotes the number of spatial locations, and D = ∑ i = 1 4 C i is the total channel depth. This sequence is then processed by a Bidirectional S6 Layer (Bi-Mamba), which applies two parallel selective state-space models—one forward and one backward—to capture long-range dependencies in both spatial directions. The output sequence z ′ ∈ R L × D is reshaped back into a 3D tensor F mamba ∈ R H 32 × W 32 × D .

To preserve the identity of the finest-scale representation while incorporating cross-scale context, we extract from F mamba only the channels corresponding to the original F 4 (i.e., the last C 4 channels), denoted as F mamba ( 4 ) ∈ R H 32 × W 32 × C 4 . This is added element-wise to the original F 4 to form the final output (Equation 5): F out = F 4 + F mamba 4 . (5)

This design ensures that the CSM block enhances the discriminative power of the coarsest-scale features through globally aware, cross-resolution context modeling—without disrupting the hierarchical structure or introducing excessive computational overhead. The use of Bi-Mamba enables efficient, linear-complexity aggregation across all scales, making the CSM block particularly suitable for histopathology analysis.

The Frequency-aware Global-Local Transformer Block (FGT, Figure 3) is the core building block of CRC-Former’s backbone, designed to jointly model global contextual dependencies and multi-directional local texture patterns via Haar wavelet-guided attention. As depicted in Figure 3, each FGT block processes an input feature map F in ∈ R H ′ × W ′ × C at a fixed spatial resolution ( H ′ , W ′ ) and produces an output F out ∈ R H ′ × W ′ × C through a hierarchical frequency decomposition and fusion pipeline.

First, the input feature map undergoes Layer Normalization, followed by a Haar Wavelet Transform that decomposes it into four orthogonal subbands, which is given by Equation 6: F L L ,   F H L ,   F L H ,   F H H = WT F in , (6) where F L L ∈ R H ′ 2 × W ′ 2 × C represents the low-frequency approximation (coarse tissue structure), while F H L , F L H , F H H ∈ R H ′ 2 × W ′ 2 × C encode horizontal, vertical, and diagonal high-frequency details (e.g., glandular edges, nuclear borders), respectively. Each subband is then processed independently by a dedicated multi-head self-attention (MSA) module with spatially constrained windowing (Figure 4):

W B -MSA: Applies big-window MSA over the entire F L L to capture long-range global context;

These frequency-specific attention outputs are concatenated and passed through an Inverse Wavelet Transform to reconstruct a fused feature map F fused ∈ R H ′ × W ′ × C . A residual connection from the original F in is added before applying Layer Normalization and a Feedforward Network (FFN), yielding the final output of the process (Equation 7): F out = FFN LN F fused + F in . (7)

To further enhance representational capacity, the FGT block is stacked N times within each stage, enabling progressive refinement of both global architecture and local texture cues. The use of sliding-window (denoted by prefix “S”) ensures efficient computation while preserving orientation-sensitive discriminative power, critical for distinguishing subtle histopathological phenotypes in colorectal cancer.

To ensure a fair and reproducible comparison with prior works Liu S. et al. (2024), we strictly adhere to the same training protocol throughout our experiments. Input patches are resized to 224 × 224 pixels, a standard resolution widely used in vision-based histopathology analysis. During preprocessing, we apply only minimal augmentation: random horizontal flipping, and channel-wise normalization using ImageNet-derived statistics ( μ = [ 0.485 , 0.456 , 0.406 ] , σ = [ 0.229 , 0.224 , 0.225 ] ) Deng et al. (2009). No advanced augmentation strategies are employed, thereby isolating architectural improvements from data manipulation effects. The model is optimized using the Adam optimizer Adam (2014) with an initial learning rate of 1 × 1 0 − 4 . The learning rate is decayed over time via cosine annealing without restarts, where the period parameter T max is set to 10 epochs. Training proceeds for a fixed budget of 300 epochs with a batch size of 32, striking a balance between memory constraints and gradient stability. All experiments are implemented in PyTorch Paszke et al. (2019) and executed on a single NVIDIA A100 GPU with 40 GB of memory. We report results averaged over three independent runs with different random seeds to account for stochastic variability.

To ensure a comprehensive and robust assessment, we employ multiple complementary evaluation metrics. Specifically, we report four standard classification metrics: Accuracy (Acc) Powers (2020), AUC Bradley (1997), Precision, Recall Baeza-Yates et al. (1999), and F1 score Sokolova and Lapalme (2009). Accuracy reflects the overall proportion of correctly classified samples and remains a fundamental indicator in diagnostic tasks. Precision and Recall, widely used in medical AI, assess different aspects of predictive quality: Precision measures the fraction of true positive predictions among all samples predicted as positive, while Recall quantifies the model’s ability to identify all actual positive cases. Since these two metrics often exhibit a trade-off—improving one may degrade the other—we further adopt the F1 score, defined as their harmonic mean, to provide a balanced evaluation of classification performance. In addition, we evaluate model discriminability using the receiver operating characteristic (ROC) curve and the corresponding area under the curve (AUC), which offer threshold-invariant measures of diagnostic capability across all classes. These evaluation metrics are summarized as the following Equations 8–12: Accuracy ACC = TP + TN TP + FP + TN + FN , (8) Precision = TP TP + FP , (9) Recall = TP TP + FN . (10) F 1  Score = 2 ⋅ Precision ⋅ Recall Precision + Recall . (11) AUC = ∑ i ∈ positiveClass rank i − M 1 + M / 2 M ⋅ N , (12) where M is the number of positive samples, N is the number of negative samples, and rank i denotes the rank of sample i based on the model’s predicted probability (sorted in descending order).

Table 1 presents a comprehensive comparison of our proposed CRC-Former against the representative architectures including ResNet101 He et al. (2016), EfficientNet Tan and Le (2019), ViT-B Dosovitskiy (2020), Swin-S Liu et al. (2021), ConvNext Liu et al. (2022), InceptionNext Yu et al. (2024), TransXNet Lou et al. (2025), BiFormer Zhu et al. (2023), GroupMixFormer Ge et al. (2023), Eff-CTM Liu S. et al. (2024), MedMamba Yue and Li (2024) and SBTAYLOR-KAN Fatema et al. (2025), across five standard classification metrics: Accuracy, F1 score, Precision, Recall, and AUC. As shown, CRC-Former achieves state-of-the-art performance on the Chaoyang colorectal histopathology dataset, outperforming all baseline models in every metric.

Specifically, CRC-Former attains an accuracy of 87.42%, surpassing the previous best (Eff-CTM, 86.30%) by +1.12 percentage points. More importantly, it demonstrates superior discriminative power through its balanced high scores across all metrics: 83.11% F1 score, 82.84% precision, 83.33% recall, and 89.67% AUC. Notably, the model’s high recall (83.33%) indicates strong sensitivity to malignant tissue regions—including early adenomas and poorly differentiated carcinomas—while its precision (82.84%) reflects reliable suppression of false positives (e.g., misclassifying inflamed or hyperplastic mucosa as cancer). The macro-averaged F1 score further confirms robustness across all four diagnostic classes, critical for real-world deployment where class imbalance is common. Compared to recent transformer-based approaches such as BiFormer (83.12% Acc) and GroupMixFormer (85.09% Acc), CRC-Former’s consistent gains suggest that its hybrid design—integrating frequency-aware attention and cross-scale state-space modeling—is particularly effective at capturing the multi-scale, heterogeneous texture patterns characteristic of colorectal neoplasia. Unlike CNNs or pure transformers, which often prioritize either local edges or global context, CRC-Former explicitly decomposes tissue morphology into interpretable frequency subbands via Haar wavelets, then fuses them adaptively using Mamba’s selective state-space mechanism. This enables the model to simultaneously resolve fine nuclear atypia (high-frequency bands) and glandular architectural distortion (low-frequency bands)—features that are often missed by conventional architectures.

Moreover, the elevated 89.67% average AUC score (the ROC curves and AUC scores of the four specific classes in the dataset are shown in Figure 5) underscores the model’s ability to maintain high discriminative power across varying decision thresholds—a key requirement for clinical screening systems aiming to minimize both false negatives (missed cancers) and false positives (unnecessary biopsies). When viewed holistically, CRC-Former’s dominance across all metrics validates its capacity to generalize beyond simple patch-level classification: it learns clinically meaningful representations that align with pathologist reasoning—balancing specificity, sensitivity, and interpretability.

In summary, these results demonstrate that CRC-Former not only advances the state of the art in colorectal cancer classification but also offers a scalable, architecture-driven solution tailored to the unique challenges of gigapixel histopathology. Its integration of signal processing priors with modern sequence modeling opens new avenues for developing robust, efficient, and explainable AI tools for digital pathology.

To validate the effectiveness of our architectural components—and specifically to assess whether attention in the frequency domain offers advantages over conventional spatial-domain sliding-window mechanisms—we conduct an ablation study using the standard Swin Transformer Small (Swin-S) Liu et al. (2021) as the baseline. Swin-S employs spatially localized, window-based self-attention and serves as a strong representative of current state-of-the-art hierarchical vision backbones.

As shown in Table 2, the Swin-S baseline achieves 85.13% accuracy and 87.44% AUC on the Chaoyang dataset. Replacing its spatial sliding-window attention with our Frequency-aware Global–Local Transformer Block (FGT)—which applies orientation-specific sliding windows ( S H , S V ) and multi-scale windows ( W B , W S ) in the Haar wavelet subbands—yields a notable improvement to 86.69% accuracy and 89.01% AUC (Model B 2 ). This gain demonstrates that decomposing features into frequency subbands and applying structure-aware attention within each band enables more discriminative modeling of histopathological textures (e.g., horizontal crypt alignment in F H L , vertical stromal invasion in F L H ) than uniform spatial windows. Separately, augmenting the Swin-S backbone with the Cross-Scale Mamba Block (CSM)—which fuses features from all four stages via selective state-space modeling—improves performance to 86.05% accuracy and 88.62% AUC (Model B 1 ), confirming the value of efficient, long-range cross-resolution context aggregation. The full CRC-Former, integrating both FGT and CSM, achieves the best results: 87.42% accuracy, 83.11% F1 score, and 89.67% AUC. Crucially, the consistent superiority of FGT over Swin-S provides direct evidence that frequency-domain sliding-window attention is more effective than its spatial counterpart for capturing the multi-orientation, multi-scale morphological signatures of colorectal neoplasia. The complementary gains from CSM further indicate that enhanced local representation must be coupled with global cross-scale reasoning to maximize diagnostic performance. This ablation study not only quantifies the contribution of each module but also establishes a key insight: leveraging wavelet-based frequency decomposition as an inductive bias for attention design leads to more pathology-aware feature learning than purely spatial mechanisms—a finding with broader implications for vision transformers in the field of medical image analysis.