The early diagnosis and timely intervention of a brain tumor case are heavily reliant on the precise segmentation of tumors from MRI scans. The manual process of segmentation comes with overly optimistic promises, as it is both labor-intensive and heterogeneous in nature (inter-rater reliability). For these reasons, automated deep learning models are gaining popularity.

SegNet is a fully convolutional network that has been used in the automated segmentation of necrosis, edema, and enhanced tumor regions alongside other multi-modal MRI sequences (T1, T1ce ( T1-weighted contrast-enhanced MRI), T2, and FLAIR (Fluid-Attenuated Inversion Recovery, a specific type of magnetic resonance imaging (MRI) sequence), like a tumor’s substructures. Researchers reported impressive F-measure outcomes of 0.85, 0.81, and 0.79 for whole, core, and enhancing tumors, respectively (5).

In the same way, a Mask R-CNN with DenseNet-41 backbone was created to perform the segmentation and categorization of tumors simultaneously, thus achieving an improved tumor boundary precision through transfer learning (6).

The more recent segmentation work conducted by Kamnitsas et al. (2023) introduced a dual-pathway 3D convolutional neural network (CNN) ensemble for high-resolution multi-view segmentation, which was integrated with CRF (Conditional Random Fields) post-processing for spatial coherence, thus achieving excellent results on the BraTS dataset.

Some studies have focused exclusively on classifying the tumors based on the specific features given. A 23-layer CNN was trained on a multi-class MRI dataset with 3,064 and 152 images for “case-based” and “control” groups, respectively, thus demonstrating the power of CNNs on large datasets and their weaknesses on smaller ones (7). To enhance the performance on smaller datasets, the model applied transfer learning using VGG16.

Another study integrated min–max normalization and dropout layers into EfficientNet for the multi-class classification of pituitary tumor, meningioma, glioma, and no tumor to enhance performance and mitigate overfitting (8).

YOLO-based architectures have also been adopted for classification due to their efficiency in real-time object detection. Studies using YOLOv5 and YOLOv7 have reported over 99% accuracy in classifying meningioma, glioma, and pituitary tumors using MRI datasets from King Khaled University Hospital (9, 10).

A recent benchmark study conducted by Karthik et al. (11) (as mentioned in Table 1 ) compared YOLOv5, YOLOv6, and YOLOv7 and reported 87.9% classification accuracy with YOLOv7 which outperformed earlier versions and even the classical methods like Faster R-CNN with VGG16.

Integrated frameworks that perform segmentation and classification in tandem have been proposed by multiple researchers. Tumors were segmented and classified using the FAHS-SVM (Fully Automatic Heterogeneous Segmentation using Support vector machine) method, which applied deep learning as well as structural and morphological information (13).

In a different approach, YOLOv5 was applied in a two-step pipeline for real-time brain tumor segmentation and classification, achieving an 85.95% detection rate (14).

The study by Bhanothu et al. (2020) used Faster R-CNN and VGG16 for detection and classification in parallel and obtained classification accuracy of 89.45% for meningioma, 75.18% for glioma, and 68.18% for pituitary tumors (15).

More recent work by Isensee et al. (2024) utilized nnU-Net with dynamic adaptation for both segmentation and classification across multiple datasets and offered improved generalizability across tumor types and institutions.

“Black-box” issues have been associated with deep learning models despite the advanced accuracy associated with them, which has brought much criticism. Very few research studies have focused on this issue.

One remarkable attempt utilized Grad-CAM-based visualization approaches to highlight attention focus areas in MRI scans during tumor classification by the models. This method enhances clinical trust and provides some level of interpretability of AI decisions.

In another study, attention-based CNNs were used to increase transparency by assigning importance weights to different MRI regions, thereby enabling clinical validation and aiding radiologists in identifying diagnostic features.

To improve the model’s accuracy and efficiency, hybrid models coupled with refinement methods have been introduced. In one of the studies, the parameters of a CNN were optimized with an Adaptive Dynamic Sine-Cosine Grey Wolf Optimizer alongside Inception-ResNetV2 for improved feature extraction and convergence rate (16).

Another work proposed a hybrid brain tumor classification (HBTC) model that combined handcrafted features from MGLCM (Modified Gray Level Co-occurrence Matrix) with deep learning for improved classification performance using a tree-based classifier for final voxel-level labeling (17, 18).

Transfer learning approaches were also explored extensively using pre-trained weights from the COCO dataset to train YOLOv4-Tiny models on the RSNA-MICCAI BraTS 2021 dataset (19), thus resulting in faster convergence and better generalization.

Despite the attempts made to apply DL models such as YOLO and CNNs or even hybrid models for the detection and classification of brain tumors, there still remain significant issues and gaps in these models, as detailed below.

Some of the commonly used models for segmentation in medical imaging are U-Net, DeepLabV3+, and Attention U-Net. These models target pixel-level identification and perform well on medical imaging segmentation tasks. However, these attention models can be slower than YOLOv7 in achieving real-time results due to the high computations required for segmentation processing.

Transformer-based models such as Swin Transformer or ViT are becoming more common for segmentation tasks. However, they are slower to train and more demanding in terms of memory utilization, thus making them less than ideal in limited-resource settings.

Therefore, while the newer versions of YOLO or other architectures may offer some incremental improvements, YOLOv7 was selected for our specific MRI scan dataset for its ability to offer a good balance between real-time performance, high detection accuracy with fast inference speed, and ease of its adaptation to our specific needs. YOLOv7 provided better accuracy in detecting and classifying brain tumors, especially small or irregularly shaped ones, as compared to other models like YOLOv5, YOLOv8, U-Net, and Faster R-CNN.

The evaluation of visualizations of brain tumors is challenging due to the size, shape, and positioning of the disorders. Scholars have come up with different ways to identify the anomalies in data, and each has its own advantages and disadvantages. Various machines are capable of producing images of brain tumors with differing levels of contrast, sharpness, number of slices, and spatial resolution. Here, we discuss the scientific specifications and architectural design of the algorithmic framework for the efficient and accurate image-based detection of brain tumors. With the recommended approach, we aim to differentiate malignant tumors in MRI scans with precision. Figure 1 showcases the preprocessing, training, and evaluation steps conducted on images containing tumors. Considering YOLOv7’s proven performance for detecting brain tumors, this study selected it as the primary framework.

To validate the accuracy of our findings, we used an MRI dataset containing 2,870 brain images sourced from Roboflow (20). MRI scans provide the highest accuracy possible in identifying brain tumors, which is why they have been included in this set. Our dataset, comprising brain tumors, had four subsets, as follows: no tumor (327 images), meningioma (823 images), pituitary tumor (834 images), and glioma (886 images). For this particular analysis, we selected approximately 70% of the entire dataset, equaling 2,009 MRI scans for training, 10% (287 MRI images) for testing, and 20% (574 MRI images) for validation.

As a means of preparing the dataset to be suitable for classification tasks, a number of preprocessing steps were performed to standardize the images of brain tumors. Below is a list of the preparatory processes that were performed: RGB images underwent grayscale conversion in order to create a single monochrome image. This can simplify the image data, which in turn will reduce computational requirements. All images were scaled to a uniform resolution of 608 × 608. This step ensured uniformity for all images before subsequent processing steps were performed. Uniform size and proportions were also maintained for the input MRI images during the preprocessing stage using scaling and aspect ratio modification techniques, which helps maintain consistency across the dataset while minimizing distortions, increasing reliability for model input.

In our work, we enhanced the architecture of YOLOv7 by integrating the CBAM (Convolutional Block Attention Module) attention mechanism along with the Spatial Pyramid Pooling Fast Plus (SPPF+) module, as they significantly optimize the feature extraction. CBAM mainly focuses on spatial and channel information, making the model more attentive to the delicate and subtle features of the tumors. Also, the SPPF+ module enriches the multi-scale context that aids in the detection of small or irregular-shaped tumors on MRI images. All these modules, when combined, result in the improved accuracy of the model in various complex situations.

Integrating attention mechanisms and SPPF+ into YOLOv7 significantly enhances the model’s feature extraction capability from complex MRI images. Attention mechanisms like SE (Squeeze and Excitation) blocks or CBAM allow the network to devote attention to the most spatially and channel-wise pertinent features. This is useful when identifying small or irregularly shaped tumors, which may resemble surrounding tissues. Also, SPPF+ excels at multi-scale local and global context feature representation through enlarged receptive fields and thus captures spatial and contextual information. These enhancements allow for better tumor detection and classification.

YOLO is a real-time object identification method that utilizes artificial intelligence. This method is well-liked since it is fast and accurate. The YOLO algorithm is important for the reasons listed below:

The methods used by YOLO are Intersection over Union (IoU), bounding box regression, and residual blocks. In order to accurately diagnose a brain tumor, the therapy needs to be stage-specific and timely. As illustrated in Figure 2 , the YOLOv7s model architecture is composed of two primary elements: the head network and the backbone network. We conducted the initial steps of image processing on the first input image to make it suitable for the backbone network.

Once the images are properly processed, the backbone network is responsible for retrieving the relevant information. These features are sent to the head network, which integrates them for further analysis toward fusion-based object detection. A balanced architectural structure to achieve detection and spatial precision that overlaps features onto the brain’s natural logic division must exist within the brain to permit the combination of multiple features efficiently and logically.

The YOLOv7 algorithms’ infrastructure system is composed of the following elements: MaxPool1 (MP1), the extended efficient layer aggregation network (E-ELAN), and the CBS (convolution, batch normalization, and SiLU (Sigmoid Linear Unit)) component. This subsystem executes the SiLU (Sigmoid Linear Unit). activation function, batch normalization, and convolution as processes to sharpen the learning ability of the network. The E-ELAN component improves the gradient flow issues within the ELAN design by enabling modular computation feature learning, which allows the network to learn new features, thereby improving the modular computation feature learning.

The MP1 component is divided into two separate sections. The shorter subdivision utilizes a 1 × 1 flow and kernel CBS method to decrease it to scale, a 2 × 2 flow and a 3 × 3 kernel to minimize each dimension of the perception, and a combination work to combine the characteristics retrieved from both divisions. The higher section utilizes a 128-output channel CBS module. An image’s dimensions are maintained as the total number of channels is decreased using the MaxPool method and the 128-output-channel CBS module. MaxPool and CBS processes improve an underlying network’s ability to recognize significant characteristics through a source visualization. Whereas the CBS method gathers areas with the least numbers, the MaxPool method collects limited localized areas with the greatest numbers. These techniques optimize the framework’s entire performance and effectiveness by improving its feature extraction capabilities.

YOLOv7’s core architecture employs the E-ELAN and incorporates the Feature Pyramid Network (FPN) design for feature extraction across multiple base layers. Utilizing the Spatial Pyramid Pooling (SPP) architecture, the Convolutional Spatial Pyramid (CSP) model enhances the collection of features at cheap computations across multiple sizes. By merging SPP and CSP, the SPPCSPC component increases the sensing area of the entire system.

Hardware specification—CPU: AMD Ryzen 9 7950X or Intel Core i9-13900K.

Integrating the ELAN-W layer improves feature extraction greatly. The MP2 block is used with two additional output channels, which are equivalent to the MP1 block. Utilizing a 1 × 1 convolution to calculate the classification, confidence, and anchor framework, the Rep structure modifies the number of image layers in its final characteristics. The Rep structure, which is based on RepVGG, includes a modified residual architecture that convolutionally decreases real estimations. Its ability to foresee is preserved even as its complexity drops.

YOLOv7’s methodology is grounded in convolutional neural networks and real-time object detection principles. Its loss function is a composite of the following:

To optimize the efficiency of a deep learning model, one must fine-tune the specific hyperparameters related to it. In this study, the hyperparameters of the YOLOv7 model were tuned to maximize detection and classification accuracy while minimizing resource expenditure. Their values, alongside a brief description of the reason for their selection, are summarized in Table 2 .

The learning rate was 0.01, facilitating incremental weight adjustments to promote stable learning and convergence. The selected batch size of 16 optimized training efficiency while accounting for GPU memory limitations. Training the model for 50 epochs achieved a balance between training duration and performance, allowing for sufficient learning while mitigating the risk of overfitting.

The Adam optimizer was utilized because of its adaptive learning rate mechanism, enhancing convergence in non-convex optimization problems. Regularization was implemented through a weight decay of 0.0005, which reduced overfitting while maintaining model complexity.

To further boost model performance on the training images, various data augmentation practices such as rotation, scaling, and flipping were implemented. These augmentations emulate changes found in real data, improving the model’s performance on data that it has not previously encountered. Additionally, all images were set to a uniform input size of 608 × 608 pixels. This standardization improves consistency and ensures the preservation of essential details required for precise tumor detection.

Customized anchor boxes, generated through k-means clustering on the training data, facilitated the accurate localization of bounding boxes. The IoU threshold of 0.5 was employed to establish valid detections, effectively balancing sensitivity and specificity in tumor detection.

The model was carefully designed to avoid overfitting to larger tumors and underfitting on smaller ones by incorporating effective regularization, data augmentation, and architectural enhancements like SPPF+. These strategies ensured balanced learning across different tumor sizes, improving generalization and maintaining high detection accuracy.

Table 3 displays the results of YOLOv7’s performance evaluation on 217 labeled box images. The precision metric was applied to pituitary brain tumors, meningiomas, gliomas, and no tumors. The model can precisely identify regions of interest (ROIs) in images according to precision measurements. The precision score of YOLOv7 was 0.837 across all classes. YOLOv7 obtained the greatest precision score of 0.909 for meningioma. The precision–confidence curves are displayed in Figure 3a , which illustrates how well the models performed.

Table 3 displays a recall evaluation of the performance of YOLOv7 on 217 labeled box images. The recall score of YOLOv7 was 0.813 across all classes. YOLOv7 had the highest recall scores (0.96 for box detection) for meningioma and a score of 0.551 for no brain tumors. The box recall–confidence curves of YOLOv7, as shown in Figure 3b , indicate that YOLOv7 performed well for meningiomas. In contrast, as shown by the green line in the figure, the recall–confidence curve for no tumor implies poor performance of YOLOv7.

With 217 labeled box images, Table 3 displays the mAP with the versions of YOLOv7 assessed at the IoU threshold of 0.5. For box detection, the mAP score was 0.879. YOLOv7 had the greatest mAP scores (0.974) for box detection in meningioma cases. YOLOv7 had 0.948 for glioblastoma, 0.929 for pituitary tumors, and 0.665 for no brain tumors. The F1-confidence curves for YOLOv7 are shown in Figure 3c . The meningioma models functioned well. Nonetheless, the lack of tumor exhibits a low slope, signifying inadequate performance in comparison with comparable categories. Moreover, the data demonstrate that for YOLOv7, the optimal box confidence value was 0.322 for obtaining an F1 score of 0.88.

Table 3 presents the mAP results obtained using the YOLOv7 algorithm for detecting the bounding boxes of 217 labeled images at an IoU of 0.5–0.95. The model considers an object detected if its IoU threshold score is between 50 and 95. For box detection tasks, the model yielded a mAP@0.5:0.95 score of 0.442. The highest mAP@0.5:0.95 score of 0.52 was observed in the meningioma cases. The precision–recall curves for the YOLOv7 model are provided in Figure 3d . It indicates that, other than the no tumor category, the model performs reasonably well for all classes. Unlike the meningioma and pituitary cases, the no tumor curve tended to have a higher rate of false positives.

Observation: High precision and recall across classes and strong mAP@0.5.

Decision: Confirms that the model effectively detects and classifies tumors with minimal false positives or negatives.

The YOLOv7 standardized confusion matrix is displayed in Figure 4 , which reveals that the model performs well for all classes with the exception of no tumor, which has a high false-positive rate of 0.35.

The YOLOv7 model’s forecast is shown in Figure 5 . These were used to maximize computational effectiveness and speed up inference, making it possible to identify instances in a variety of images. The anticipated images demonstrated the accuracy with which the brain tumor identification system operated following its training on the initial images. The complete outcomes of classification using the YOLOv7 model are presented in Figure 6 . The experiment’s findings proved that YOLOv7 produced useful outcomes.

Observation: Diagonal dominance with few misclassifications, mainly between glioma and meningioma.

Decision: Model performs well, but further data augmentation or fine-tuning could reduce misclassification.

Observation: Accurate bounding boxes drawn over tumors with correct class labels.

Decision: Demonstrates the model’s potential for assisting in real-time clinical diagnosis.