To meet the requirements of high efficiency and high precision in monkeypox case identification, we define a residual convolution module based on the λ function and a residual convolution module based on the contextual transformer, as shown in Figure 2 . The design idea is to replace the 3×3 convolution layer in the traditional residual module with λ calculation layer and CoT calculation layer. These two computing layers will be detailed in sections 3.3 and 3.4.

In this paper, LaCTResNet neural network model is defined by improving the feature extraction module of the ResNet backbone network, and the overall framework of the network model is shown in Figure 3 . For the infected image of the monkeypox virus to be identified, this network finally outputs the category of infected virus predicted by the network model after passing through a convolution layer, two residual convolution modules based on the λ function, two residual convolution modules of the contextual transformer, an average pooling layer, a fully connected layer, and a Softmax layer. In the figure, Conv1 represents the ordinary convolution layer, LamResConv2 and LamResConv3 represent the λ function residual convolution module, and CoTResConv4 and CoTResConv5 represent the contextual transformer residual convolution module.

The working mechanism of residual learning (23) is realized by residual connection. There are no redundant branches in the traditional convolutional neural network. From top to bottom, the input signal will be transformed nonlinearly at each layer and directly transmitted to the next layer. The structure of residual learning includes the main branch and the residual difference branch, as shown in Figure 4 . The input signal x enters the main branch, passes through the Weight Layer and the activation function ReLU, and outputs the signal F(x); The residual differential branch directly copies the input signal x and transmits it to the output end of the main branch through the cross-layer residual connection, and adds it with the output signal F(x) of the main branch to get F(x)+x, which is transmitted to the next layer after the activation function ReLU. Through residual connection, the main branch’s output signal contains the characteristics of traditional nonlinear transformation and residual characteristics. The residual feature represents the difference between the input signal and the expected output. By adding it to the output signal of the main branch, the network can learn these differences more efficiently, improve the accuracy of network feature extraction and recognition, and effectively avoid network degradation.

Modeling long-distance information interaction is an essential topic in deep learning. At present, the mainstream paradigm is the attention mechanism. It is easy to find that the high occupation of secondary memory is not conducive to dealing with long sequences or multi-dimensional input by analyzing the calculation diagram of the attention mechanism. Considering the limitation of self-attention, we introduce the λ function layer (24) into the model, which provides a novel general framework for capturing long-range information interaction between input signals and structured contexts. The λ function layer captures the information interaction by transforming the available context into a linear function λ and applying these linear functions to each input value respectively. The attention mechanism defines a convolution kernel of similarity between input and context. At the same time, the λ function layer aggregates the context information into a linear function with a fixed size, thus skillfully solving the situation that attention tries to occupy much memory. The detailed calculation diagram of the λ function layer is shown in Figure 5 . The left part is λ function generated by context, and the right amount is λ function applied to the query.

From the above information, we can draw that, firstly, the Context gets V(Value) and K(Key) through linear projection, and the mathematical expressions are shown in formulas (1) and (2):

Next, V is multiplied with the position code E_n to obtain the position-based λ n p , and the mathematical expression is shown in equation (3):

K is then obtained by the normalization operation K ¯ with the mathematical expression shown in equation (4):

After that, K ¯ is multiplied with V to obtain the content-based λ c with the mathematical expression shown in equation (5):

Finally, λ c and λ n p are multiplied to obtain λ n containing location and context information, and the mathematical expression is shown in equation (6):

The right part is to map the input value X (X ∈ { x 1 , x 2 , x 3 , ⋯ , x n } ), by features to Q(Query), Q ∈ { q 1 , q 2 , q 3 , ⋯ , q n } . Then, λ n ( λ n ∈ { λ 1 , λ 2 , λ 3 , ⋯ , λ n } ) on the left side is applied to Q one by one. The final output Y (Y ∈ { y 1 , y 2 , y 3 , ⋯ , y n } ) is mathematically expressed as shown in equations (7) and (8):

Figure 6A shows that the traditional self-attention can trigger feature interaction in different spatial positions. However, all the attention matrices of Query and Key are realized by calculating independent query key pairs, ignoring the rich context information between keys, thus limiting the visual learning ability of the self-attention mechanism in two-dimensional images. Therefore, we introduce the Contextual Transformer Layer (CoT) (25) into the traditional residual module and construct a residual convolution module based on contextual transformation, the structure of which is shown in Figure 6B . Through the fusion modeling of local static context and dynamic global context, the CoT computing layer can enlarge the feature distance between samples, improve the problems of significant parameters and weak feature extraction ability in the original ResNet network, and realize the effective extraction of feature information.

From the information in Figure (b), it can be seen that the input feature X of size H × W × C is given, and three variables Q = X, K = X, and V = XW_V are defined (here only V is mapped to the feature, and the original X values are still used for Q and K). A grouped convolution of k × k (the value of k is taken as 3 in this experiment) is performed on K to obtain K with local context information representation (denoted as K¹ , K¹ ∈R^H×W×C ), and this K¹ can be seen as static modeling on the local information. Then K¹ and Q were Concat, and the result of Concat was subjected to two successive 1×1 convolution operations to obtain the attention matrix A. The mathematical expression is shown in equation (9):

Unlike the traditional self-attentive mechanism, the A-matrix here is obtained from the interaction of Query information and local context information K¹ , rather than just a simple correspondence between Query and Key. That is, the guidance of regional context modeling enhances the self-attention mechanism. This attentional computational graph A and V are then subjected to a matrix product operation, which yields K² for dynamic context modeling, with the mathematical expression shown in equation (10):

Then the final feature result Y is obtained by fusing K¹ from local static context modeling and K² from global dynamic context modeling. The design of the CoT layer unifies the context mining between adjacent keys and the self-attentive learning of 2D feature maps, using the context information between input keys to guide the self-attentive learning, thus avoiding the introduction of extra branches for context mining and improving the representational power of the network.

We conducted experiments on a computer with a GPU model NVIDIA GeForce GTX 3090, and the details are shown in Table 2 .

Firstly, the ratio of four groups of residual convolution modules is fixed at 3:4:6:3. That is, the module configuration ratio of ResNet50 is followed. Then, the 3×3 convolution in four groups of traditional residual convolution modules is replaced by the λ function layer or CoT layer for discussion. A total of five groups of experiments were carried out, and there were sixteen situations: in the first group of experiments, all four groups were replaced by the CoT layer; in The second group of experiments, replacing one group with λ function layer and the other three groups with CoT layer, has four situations; The third group of experiments, replacing two of them with λ function layer and the other two with CoT layer, has six situations; The fourth group of experiments, replacing three of them with λ function layer and the other with CoT layer, has four situations; In the fifth group of experiments, all four groups were replaced by λ function layer. The experimental results are shown in Table 3 .

Looking at the above table, we can see that the effect of replacing all four groups of modules with the λ function layer is not ideal, and the recognition accuracy is only 61.98%; The CoT layer replaces all of them, and the recognition accuracy is 84.06%. In the second group of experiments, the best effect is that the λ function layer replaces the first group, and the other three groups are the CoT layer, and the recognition accuracy reaches 87.37%. In the third group of experiments, the best effect is that the λ function layer replaces the first and second groups, and the CoT layer replaces the other two groups, and the recognition accuracy reaches 89.02%. In the fourth group of experiments, the best effect is that the λ function layer replaces the first, second, and fourth groups, and the CoT layer replaces the third group, and the recognition accuracy reaches 88.43%. From this, we get the optimal convolution module collocation form: the first two groups are replaced by the λ function layer, and the CoT layer replaces the last two groups. We renamed the optimal model LaCTResNet5012 as LaCTResNet50.

Next, we use the 1:1:1:1 module distribution ratio of ResNet18 and Swin Transformers’ 1:1:3:1 module distribution ratio (26) to discuss the LaCTResNet50 model further. These two ratios are proved effective in improving the model’s accuracy (23, 27), and the experimental results are shown in Table 4 .

According to the above table, the recognition accuracy of LaCTResNet2222 and LaCTResNet3393 is higher than that of LaCTResNet50, with an increase of 0.24% and 2.83%, respectively. Although the recognition accuracy of LaCTResNet2262 is the same as that of LaCTResNet50, it has certain advantages in parameters. In the end, the best model we got was LaCTResNet3393, and the recognition accuracy was 91.85% on the independent test set of the monkeypox data set. According to the overall structural characteristics of the network, we renamed LaCTResNet2222 to LaCTResNet26, LaCTResNet2262 to LaCTResNet38, and LaCTResNet3393 to LaCTResNet56.

The experimental data mentioned above show that the LaCTResNet model we designed has the advantages of both lightweight and high accuracy. In order to be able to disclose the details of the model’s framework more intuitively, we output the framework structure of the LaCTResNet series of models through the program and list the parameters in Table 5 .

To ensure the reliability and stability of the model, we evaluated the model using cross-validation experiments. For a limited sample dataset, five-fold cross-validation is commonly used to evaluate or compare the performance of models. In 5-fold cross-validation, the dataset is divided into five mutually exclusive subsets (i.e., D = D₁ ∪ D₂ ∪… ∪ D₅, D_i ∩ D_j = ∅ (i ≠ j)), where D - D_i is used as the training set and D_i (i = 1, 2,…, 5) as the validation set. The cross-validation process is repeated five times, and the results of the five times are averaged to evaluate the model’s performance. The 5-fold cross-validation of our proposed model achieves an average recognition accuracy of 91.25% on the training set and 90.54% on the test set. The standard deviations of the above experimental data are minor, only 0.0091 and 0.0119. The experimental data are shown in Table 6 .

We selected the classic deep convolution neural networks AlexNet, VGG16, Inception-V3, ResNet50, and EfficientNet-B5 as the comparative experimental models, and the experimental results are shown in Table 7 .

The above table shows that the recognition accuracy of classical network models on the independent test set of monkeypox is all below 80%. In comparison, the recognition accuracy of the LaCTResNet series models proposed by us is about 90%, which is 13.23%, 13.03%, and 15.82% higher than that of the ResNet50 model. In order to more intuitively reflect the relationship between the accuracy and parameters of each network model, we draw the relationship diagram between accuracy and parameters, as shown in Figure 7 , through visualization software. Considering these two parameters comprehensively, we find that LaCTResNet series models have advantages.

In this section, we analyze the accuracy and convergence of the loss value of the model on the monkeypox training set in detail. In the experiment, we adopt a dynamic learning rate strategy. That is, the learning rate decays by 90% every 30 training cycles. This strategy can make the model jump out of the “trap” of optimal local value and avoid local oscillation, thus effectively improving the convergence speed of the model. The two-dimensional line chart of each model’s accuracy and loss value during training is shown in Figure 8 . Observing all the curves in the diagram, we can find that the recognition accuracy of each model has significantly jumped after the 30th cycle, and the loss values have decreased significantly. After the 60th cycle, the accuracy and loss values gradually converged and approached a stable value. Let us compare the classic model and LaCTResNet series models as two groups. We can find that the recognition accuracy of the classic model fluctuates between 0.5 and 0.8 after the 60th cycle, while that of LaCTResNet series models fluctuates between 0.7 and 0.9. LaCTResNet series models have apparent advantages in convergence speed and recognition accuracy. The overall trend of the accuracy and loss value curve is consistent with the target expectation and fluctuates in the normal range, showing that the model can deal with local optimal traps, performs well in over-fitting, and can fully grasp the potential “universal law” in the sample.

In order to verify the generalization ability and robustness of the model, we conducted prediction experiments on the LaCTResNet series model on an independent test set of monkeypox images, and the confusion matrix heat map was drawn by a visualization tool, as shown in Figure 9 , which can visually demonstrate the recognition ability of the model.

In the heat map, the vertical coordinates represent the predicted values of the samples, and the horizontal coordinates represent the actual values of the samples. The main diagonal numbers represent the number of images when the predicted values agree with the actual values, i.e., the number of images correctly recognized by the model, and the remaining position numbers represent the number of images when the predicted values differ from the actual values, i.e., the number of images incorrectly recognized by the model. Observing the confusion matrix heat map of all models, we can find that the classical model has a significant difference in the color depth of the main diagonal blocks of the heat map, which indicates that the model incorrectly recognizes more images; the main diagonal blocks of the heat map of the LaCTResNet series of models all consistently show a darker color, which indicates that the series of models proposed in this paper show better recognition of all six types of images on the monkeypox data set. It has certain technical reference value and application significance.

The model’s performance can be evaluated more quantitatively by further analyzing and processing the values in the confusion matrix. In this regard, we must also define the following metrics: TP, FP, TN, and FN, representing true positive, false positive, true negative, and false negative, respectively. Among them, Precision is the proportion of positive cases correctly predicted by the model to the actual positive cases, reflecting the checking accuracy of the model, and the mathematical expression is shown in equation (11):

Recall, also known as True Positive Rate (TPR), refers to the proportion of positive cases correctly predicted by the model to all positive cases predicted by the model, reflecting the model’s check-all rate and the mathematical expression is shown in equation (12):

The False Positive Rate (FPR) refers to the proportion of positive cases that the model incorrectly predicts to all positive cases, also called the false identification rate and false alarm rate, and the mathematical expression is shown in equation (13):

The F1 Score (F1_score) is the summed average of precision and recall, which is used to measure the comprehensive performance of the model and the mathematical expression is shown in equation (14):

Since this experiment is a multiclassification problem, the macro average of the above evaluation metrics is used to measure the “global” performance of the model, which are the macro accuracy rate (macro-P), macro completeness rate (macro-R) and macro F1 score (macro-F1_score), and the mathematical expressions are shown in equations (15), (16) and (17), respectively:

Where, P_i refers to the precision of the ith class; R_i refers to the recall of the ith class. Detailed data on the recognition performance of the LaCTResNet family of models are shown in Table 8 .

Observing the information in the above table, we can find that the LaCTResNet56 model has the most significant macro-F1_score value as well as the highest F1_score value for monkeypox case recognition, the model has the best overall recognition performance, and the overall recognition of monkeypox cases is better than other models.

As shown in Figure 10 , the combined recognition ability of the LaCTResNet family of models for each category is demonstrated. The models are less effective in recognizing measles cases. It is because the clinical symptoms of measles, chickenpox, and smallpox all show a large rash on the skin with redness and swelling. The characteristics are highly similar, which can easily lead to model misidentification.

Figure 11 shows the Receiver Operating Characteristic (ROC) curve of the optimal model, LaCTResNet56, for each category of cases on the monkeypox dataset. The vertical coordinate represents the TPR, the horizontal coordinate represents the FPR, and the Area Under Curve (AUC) represents the recognition effect of the model for that category, and the more significant the area, the better the effect. If the AUC is less than 0.5, it means that the model does not have realistic reference significance for identifying this category, while the closer the AUC is to 1, the better the model is for identifying this category. It can be seen from the figure that the AUC of measles cases is 0.9235. It is worth mentioning that the AUC of monkeypox cases reaches 0.9442, and the average AUC of all categories reaches 0.9476, proving that the model can identify monkeypox cases with high accuracy.