Images depicting wheat rusts at various severity level stages were captured within the fields of ICAR-Indian Agricultural Research Institute, New Delhi, India, spanning three consecutive crop seasons. Image acquisition took place during sunny conditions, typically between 11:00 am to 1:00 pm, at ten-day intervals following the initial onset of disease symptoms. This timing ensured consistent leaf growth stages across all captured images.

A handheld mobile camera with a 20-megapixel resolution and a 25mm wide-angle lens was utilized for image acquisition. The deliberate use of mobile devices instead of professional cameras aimed to mirror the tools commonly employed by farmers in similar scenarios. The severity level estimation dataset encompasses images of three major wheat diseases, categorized into three severity levels by plant pathologists: low, medium, high and healthy ( Figure 1 ). Images were captured by directing the camera lens toward regions of the leaves exhibiting disease symptoms at various growth stages. Subsequently, pathologists labeled the severity levels based on the percentage of disease symptoms present. Images categorized as low severity level exhibited severity symptoms ranging from 0 to 25 percent, those classified as medium severity level ranged from 25 to 50 percent, and high severity level images displayed more than 50 percent disease symptoms. The images with no disease symptoms were considered healthy as shown in Figure 1 .

The primary emphasis of this study lies in low-level severity image classification to facilitate timely disease detection and mitigate crop loss. Initially, the severity stage estimation dataset comprised 5438 images distributed across ten classes, including three rust severity stages for each disease class and a class representing healthy leaves (refer to Table 1 ). To enhance classification performance and achieve balance among disease classes, the original dataset underwent augmentation, resulting in a total of 10252 images, with 1000 images allocated to each disease-infected class.

Prior to model training, image pre-processing, and augmentation were performed to enhance model performance. Initially, duplicate, out-of-focus, noisy, or blurry photos were eliminated from the dataset to ensure data quality. Subsequently, the Augmentor Python package was utilized to augment the images by employing various techniques such as zooming, flipping, and rotating. This augmentation process aimed to diversify the dataset and enrich it with variations, thereby facilitating robust model training. Each class of severity stage images was augmented to contain 1000 images, ensuring balanced representation across classes (refer to Table 1 ). Additionally, the images were resized to 256 x 256 pixels to accommodate hardware constraints, optimize computational efficiency, and enhance the model’s generalization and performance. This pre-processing and augmentation pipeline laid the groundwork for effective model training on the augmented dataset.

The research methodology is visually depicted in Figure 2 . Initially, images were captured from real-world wheat crop fields using mobile devices. Domain experts labeled each image with the corresponding type of wheat rust and its severity level, organizing them into distinct folders. Subsequently, image processing techniques, including resizing, filtering, and noise reduction, were applied to refine the raw images. Augmentation techniques, such as random rotation, translation, flipping, and zooming, were employed to diversify the image dataset and validate the models before experimentation. Two datasets were created: the original dataset containing 5438 images and an augmented dataset comprising 10252 images, both segregated into train, test, and validation sets in an 80:10:10 ratio for experimentation purposes. Initially, the performance of the fine-tuned EfficientNet B0 model was evaluated on both datasets. Subsequently, to enhance the model’s performance, the proposed model was developed by integrating the CBAM module (Tan and Le, 2020) into the fine-tuned EfficientNet B0 model. The attention mechanism’s channel and spatial modules focus on key disease symptoms, aiding in determining the severity level of wheat rust. Figure 2 illustrates the flowchart of the wheat disease identification and severity stage estimation framework, with subsequent sections elaborating on each phase of the framework.

The EfficientNet B0 serves as the foundational model within the EfficientNet family, encompassing a total of eight variants (B0-B7) ( Figure 3 ). EfficientNet B0 architecture achieves high accuracy and computational efficiency through a compound scaling approach as described by Atila et al. (2021). The EfficientNet architecture employs the Mobile Inverted Bottleneck Convolution (MBConv) as its primary building block, introduced by Sandler et al. (2018) and illustrated in Figure 3 . The MBConv block comprises three components: a 1 × 1 convolution (1 × 1 Conv), Depth-wise convolution (Depth-wise Conv), and a Squeeze-and-Excitation (SE) module. Initially, the output of the preceding layer is passed through the MBConv block, where the number of channels is expanded using a 1 × 1 Conv. Subsequently, a 3 × 3 Depth-wise Conv reduces the number of parameters, followed by channel pruning that compresses the channel count through another 1 × 1 Conv. A residual connection is then introduced between the input and output of the projection layer to enhance feature representation. The SE module, as shown in Figure 3 , incorporates two key operations: squeeze and excitation. The squeeze operation is performed using global average pooling (AvgPooling), while the excitation operation involves two fully connected layers activated sequentially with a Swish activation and a Sigmoid activation function. This design facilitates efficient parameter utilization while maintaining high performance. However, the SE module focuses on channel-wise feature recalibration by emphasizing informative channel characteristics while suppressing less relevant ones. However, this approach primarily addresses channel-specific information and overlooks spatial context, which is critical for visual recognition tasks such as severity estimation. This limitation negatively affected the model’s classification accuracy for severity estimation.

To address this, the (CBAM) was integrated into EfficientNet-B0 in place of the SE module to enhance feature extraction by simultaneously considering both channel and spatial information. The modified network, referred to as EfficientNet-CBAM, is illustrated in Figure 4 . Key modifications include the replacement of the SE module in each MBConv layer with a CBAM module, allowing the network to retain vital spatial information alongside channel-specific features, particularly for identifying disease severity symptoms. Additionally, a CBAM module was introduced after the final convolutional layer, refining the extracted features and improving the network’s classification performance. The final convolutional layer of EfficientNet B0 produces feature maps, which serve as input for the Convolutional Block Attention Mechanism (CBAM) module (see Figure 4 ).

Attention mechanisms, extensively utilized in research, augment feature extraction and boost model performance in image classification tasks (Woo et al., 2018; Wang et al., 2017). Our architectural design incorporates a convolutional block attention module with two key components: the Channel Attention Module (CAM) and the Spatial Attention Module (SAM) (refer to Figures 5 ). These two modules work together to improve feature extraction and representation within the generated feature maps (Woo et al., 2018). The input feature map W, representing the wheat rust-infected leaf image, undergoes processing within the CAM, producing the channel attention feature map Mc. This map highlights essential image information, which is then used to generate the refined feature map W’. Element-wise multiplication between M_c and W yields the improved feature map W’. Subsequently, W’ is subjected to processing within the Spatial Attention Module, generating the spatial feature map M_s. This map selectively emphasizes significant image areas. The enhanced feature map W’ is subsequently combined with the spatial feature map M_s. This multiplication yields the ultimate feature map W’’, which encapsulates the representation of the wheat rust image. The Convolutional Block Attention Mechanism operates through the following Equations 1, 2:

The CBAM module’s channel attention mechanism utilizes pooling operations to compress the feature map W, focusing solely on essential symptom regions within the image while disregarding extraneous information or features. Conversely, the spatial attention mechanism identifies significant feature locations post-CAM processing. This process involves spatial dimension compression of the feature maps W’ and the generation of the spatial attention feature M_s utilizing the sigmoid activation function. It highlights critical features within specific image area, enhancing intermediate features.

The proposed methodology employs transfer learning, where a novel model aimed at disease severity stage identification is trained utilizing a pre-trained model, EfficientNet B0, as the foundation for learning. While retaining the initial layers of the EfficientNet B0 model, the final layer is replaced with new layers. These newly introduced layers are subsequently fine-tuned to classify infected leaves into ten distinct classes using, ‘WheatSev’ dataset developed by us, as per the methodology given by Too et al. (2019). Thus, the WheatSevNet model is designed to accurately classify the severity stages of wheat rust infections by enhancing feature extraction and representation. It builds upon EfficientNet-B0, a widely used deep learning architecture known for its computational efficiency and high performance. However, EfficientNet-B0’s Squeeze-and-Excitation (SE) module, while effective for channel-wise recalibration, lacks spatial feature extraction capabilities. To overcome this limitation, WheatSevNet integrates the Convolutional Block Attention Module (CBAM) in place of the SE module within each MBConv block of EfficientNet-B0 as depicted in Figure 6 . The CBAM module consists of two components: the Channel Attention Module (CAM), which selectively enhances significant channels using global average pooling, max pooling, a multi-layer perceptron (MLP), and a sigmoid activation function, and the Spatial Attention Module (SAM), which refines feature extraction by applying average and max pooling across the channel axis, followed by a convolutional layer and a sigmoid activation function. In WheatSevNet, each MBConv block of EfficientNet-B0 is modified by replacing the SE module with CBAM, ensuring both spatial and channel-wise attention are effectively incorporated. Additionally, a final CBAM layer is introduced after the last convolutional layer to further refine feature maps before classification. The model leverages transfer learning and fine-tuning, utilizing pre-trained EfficientNet-B0 weights with the initial layers frozen while optimizing the later layers for severity classification. Further modification involves the addition of a normalization layers, fully connected (FC) layer, a dropout, and a convolutional layer, as depicted in Figure 6 for detecting and classifying different severity stages of wheat rust. By integrating CBAM at multiple levels, WheatSevNet achieves enhanced feature extraction, capturing both disease-specific spatial structures and critical channel-wise characteristics, thereby improving classification performance over traditional EfficientNet-based approaches.

Additionally, Table 2 presents the hyperparameters employed for the disease severity estimation models. Owing to low stopping, the number of epochs ranged from 27 to 50, with a fixed learning rate of 0.001. To mitigate this risk, the authors have incorporated several measures to prevent overfitting. These include data augmentation, which helps in artificially increasing the dataset size and providing more diverse training examples. The augmentation process involved applying random transformations such as rotation, flipping, scaling, and color adjustments to create new variations of the existing images. This technique aimed to simulate real-world variations in the data, which helps prevent the model from overfitting to the specifics of the training set. Furthermore, class balancing was ensured by augmenting each class of severity stage images to contain a total of 1000 images per class, ensuring that all classes were equally represented in the dataset. This balanced representation prevents the model from being biased towards any particular class, enhancing its ability to generalize across different categories. Further, a dropout rate of 0.20 was implemented during the training process. Dropout works by randomly disabling 20% of the neurons in each layer during training, which helps prevent the model from becoming overly reliant on specific features and encourages it to learn more generalized patterns. Additionally, L2 regularization has been applied to control the complexity of the model and prevent it from overfitting to the training data. In addition to these measures, early stopping with patience as 3 was incorporated as an extra safeguard against overfitting. This technique monitors the validation loss during training, and if no improvement is seen for a specified number of epochs (the patience parameter), the training process is halted early. This prevents the model from continuing to learn noise and overfitting to the training data. The patience parameter was set to allow the model to train for several epochs without improvement before stopping, ensuring that it had enough time to converge but also preventing unnecessary overfitting. All these hyperparameters for the severity estimation models are reported in Table 2 . Categorical cross-entropy served as the loss function during model training, while the batch size for experimentation was fixed at 32. The subsequent subsection will address the third objective of the research study, focusing on elucidating the validation of the developed models and their integration into mobile applications.

During the experiment, various pre-trained classical deep-learning models were compared to the proposed model. These models included VGGNet (Simonyan and Zisserman, 2014), ResNet152 (He et al., 2016), InceptionV3 (Szegedy et al., 2016), MobileNetV2 (Sandler et al., 2018), and DenseNet121 (Huang et al., 2017). These models underwent parameter resetting before training, followed by modifications to the bottom layers of the pre-trained networks. The bottom layer was substituted with a new SoftMax and output layers containing ten severity stage classes from the datasets.

The experimentation was conducted on a robust DGX server featuring GPU capabilities, with computations executed using the Keras and TensorFlow frameworks. The system has Ubuntu as the operating system, supported by an Intel^® Xeon^® CPU. All computationally intensive tasks were handled by the NVIDIA Tesla V100-SXM2 GPU, boasting ample memory resources of 528 GB (refer to Table 3 ).

The accuracy of classification predictions in machine learning experiments is assessed through confusion matrices. It is used to analyze the correspondence between the predicted and actual prediction scores for individual classes of a classification model. Other metrics such as precision, accuracy, F1 score, and recall are also used to assess the performance of our model. Precision describes the ratio of true positives to all positive predictions, while accuracy refers to the proportion of correctly identified predictions relative to the total number of the predictions. On the other hand, Recall measures the ratio of true positive cases to all positive predictions (Hossin and Sulaiman, 2015). The F1 score is further calculated using the harmonic mean of precision and recall, providing a balanced evaluation of the model performance (Equations 3–6).

The “true positive” (TP) represents the count of images accurately detected within each severity stage class. Conversely, “true negative” (TN) represents the overall number of images correctly identified across all severity stages, excluding the specific severity stage to which they belong. “False negative” refers to the count of images wrongly classified within each relevant severity class, while “false positive” (FP) indicates the number of images incorrectly classified as belonging to different severity stage classifications. Finally, the predictive performance of the proposed model is summarized by the F1 score.

Figure 8 illustrates the confusion matrix of the severity estimation model, depicting ten classes based on actual and predicted labels from augmented datasets. The numbers along the diagonal signify accurately identified images, while those outside the diagonal indicate instances of misclassification (Ting, 2017). Specifically, among the four low severity images of brown Rust, two were erroneously identified as medium-stage severity, and one as healthy ( Figure 8 ). Similarly, misclassifications occur in other severity stages of brown Rust, with images of high and medium levels misclassified as medium and low, respectively. Furthermore, low-severity images of brown Rust were mistakenly classified as medium severity and healthy. Additionally, misclassifications were observed in stem rust and yellow rust severity stages.

Upon analyzing the confusion matrix of EfficientNetB0 embedded with CBAM, it was noted that Rust spread on the upper surface of the leaf leads to significant confusion between the low and medium severity stages of the disease. Referencing Table 5 , the classification report derived from the confusion matrices includes F1, precision, and recall metrics for the proposed disease severity estimation model. A model is deemed appropriate if its F1 score approaches one. After evaluating these performance metrics, the following findings emerge: On the augmented dataset, the average precision, recall, and F1 score for identifying severity stages in brown rust and stem rust is 97%, whereas, for yellow Rust, the average score for precision, recall, and F1 measure is 95%.

Table 5 shows that augmented datasets yield superior results compared to non-augmented datasets across all diseases and their categories. However, for stem rust disease, all performance parameters in non-augmented datasets surpass 90% across all stages, possibly due to easily identifiable features of stem rusts. Conversely, for brown and yellow Rust, the performance of non-augmented datasets is notably inferior to augmented datasets. Another noteworthy finding pertains to the higher classification accuracy of healthy leaves, which can be attributed to (i) the more significant number of images (1252) and (ii) the absence of any classes for healthy leaves. Although the accuracy improves as the disease stage matures, even in low stages, precision exceeds 90% for all types of rusts. For the augmented dataset, precision reaches 97% for brown Rust and 91% for yellow Rust.

Figure 9 depicts the training and validation curves for the wheat severity estimation model, offering insights into the learning process. Notably, the accuracy curves indicate that our proposed severity model serves as a commendable and efficient fit model (Hossin and Sulaiman, 2015). The findings of the disease severity model underscore its efficacy in automatically identifying wheat severity stages based on images.

In conclusion, the proposed models prove useful for identifying diseases at a low severity level, as evidenced by the accurate identification of most images in the low stages of Rust. This early severity assessment holds promise for crop preservation and can significantly minimize crop loss.

The interpretability of the proposed model is carried out using the GradCAM (Selvaraju et al., 2020). Figure 10 illustrates that the proposed severity estimation model focuses explicitly on the features and the symptoms that play an important role in identifying the severity level of the type of the wheat rusts. The activation maps shown in the figure facilitate specific regions in the input test images necessary for estimating the severity of the disease. Thus, we aimed to illustrate how the model is directing its attention towards the areas where the symptoms are most noticeable in order to identify the disease at a low stage of severity. The attention mechanism has been found to improve the model’s ability to identify the appropriate symptoms in the correct location accurately.

In this study, we also developed an Android-based mobile application as a practical tool for the automatic identification of wheat diseases and their corresponding severity stages in agricultural fields. The proposed model was integrated into the application’s backend to facilitate this functionality. The mobile application allows users to capture real-time images from agricultural fields or select images stored in their mobile device gallery. These images are subsequently uploaded to a server for analysis. The analytical process begins with the application determining whether the uploaded image depicts a healthy or diseased wheat plant. For images identified as healthy, the result is directly displayed as “healthy”. Conversely, if the image is diagnosed as diseased, the application proceeds to identify the specific type of disease. Following disease identification, the application further evaluates the image to estimate the severity stage of the detected disease. Figure 11 provides a detailed illustration of the application’s process flow, from image acquisition to disease identification and severity stage estimation.

The developed mobile application for wheat disease severity estimation follows a streamlined workflow Figure 12 . Upon launching, a splash screen introduces the app, followed by an interface that allows users to capture or upload a wheat leaf image. Once an image is uploaded, the “Identify” button determines if the leaf is healthy or diseased. For healthy images, the app displays a message indicating no further action is required. If a disease is detected, the application identifies the disease type and provides an option to predict its severity stage. The final screen presents the identified disease along with its severity stage, providing a complete diagnostic result for the uploaded image.

The Table 6 offers a comprehensive overview and comparative assessment of various models utilized for disease classification across different crops, alongside their respective training or testing accuracies. Our proposed model, specifically for diagnosing three major rusts in wheat crops, achieved a commendable testing accuracy of 96.68%. When compared with existing literature, our model emerges as a strong contender, demonstrating competitive performance. Notably, existing models developed for crops such as apple, tomato, coffee, cucumber, and grape achieved accuracies ranging from 90.4% to 97.75%, albeit focusing on single disease classes. Importantly, prior attempts at wheat severity estimation encompassing three diseases and their severity levels were scarce. Despite this, our model’s accuracy not only matches but also exceeds the reported accuracies in the literature, underscoring its efficacy in wheat rust severity estimation.

In this study, our primary contributions are twofold: Firstly, we curated a robust dataset for wheat disease classification, encompassing the estimation of severity categories. The non-augmented dataset comprises 5438 images, while the augmented dataset boasts a total of 10252 images. This dataset lays a strong foundation for future research in this domain. Secondly, we introduced WheatSevNet, an algorithm capable of identifying various wheat disease categories and assessing their severity levels. Despite the challenges posed by multi-disease classes and multi-severity levels, our enhanced model achieved an impressive accuracy rate of over 96%. This performance is comparable even to other algorithms designed for single disease identification. We could not compare with wheat severity estimation models for three diseases as no such published attempt is available to the best of our knowledge. The success of our approach not only addresses an immediate need in agricultural research and opens up promising avenues for future investigations in this field. We hope our study will inspire further exploration and innovation in automated plant disease diagnosis and severity estimation.