Under natural light conditions, using iPhone seven Plus mobile phone (IOS 10.1 operating system, F/1.8 wide-angle, and F/2.8 telefocal dual-lens camera, 12 million pixels), the rice disease images are captured in the rice experimental field of Heilongjiang Academy of Agricultural Reclamation Sciences, China. The mobile phone is fixed on a bracket, using automatic exposure mode, taking a picture at a moderate distance from the diseased rice leaves. The photo size taken by the phone is 3,024 × 4,032 pixels. The images of three rice diseases were obtained in the early, middle, and late stages, respectively. In each cycle, 500 images were collected for each of the three rice disease images. Therefore, a total of 5,600 sample images were collected, including 1,500 images of rice blast, sheath blight, and brown spot, of which 1,100 were uninfected. Each image is stored in the PNG format and compressed into a uniform size of 400, ×, 300. An example of the original rice disease image sample is shown in Figure 1; 70% of the samples are randomly selected from the data set as a training set, and the remaining 30% of the samples are chosen as a testing set. The data set division is shown in Figure 2.

The feature extraction of the disease spot image is a key step in rice disease recognition. According to the spot characteristics of rice sheath blight (taking rice sheath blight as an example), three features of color, texture, and shape are extracted from the spot image.

In Figure 5, an energy model called restricted Boltzmann machine (RBM) is shown. It is a special kind of stochastic neural network model that owns a two-layer architecture. The input layer is also called the visible layer, in which visible units represent observations. The other layer is called the hidden layer, in which hidden units represent features. Visible variables S are linked to hidden variables D through undirected weighted connections W. There are no connections in the same layer. This constitutes a bipartite structure graph. The parameters are introduced as follows:

S = ( s 1 , s 2 , … , s m ) ∈ R m , where m is the neurons’ number in the visible layer.

D = ( d 1 , d 2 , … , d n ) ∈ R n , where n is the neurons’ number in the hidden layer.

A = ( a 1 , a 2 , … , a m ) ∈ R m represents bias in the visible layer.

B = ( b 1 , b 2 , … , b n ) ∈ R n represents bias in the hidden layer.

W = ( w i , j ) ∈ R n × m stands for the weight matrix that connects the visible layer and the hidden layer. w _i,j is the weight that connects the jth neuron in the hidden layer and the ith neuron in the visible layer.

ϱ = (W, A, B) represents the unknown model parameter.

RBM is an energy based model. Assuming that the variables of both visible and hidden layers obey Bernoulli distribution, the joint probability distribution of visible and hidden variables is given by energy function. For a given state (s, d), an energy function can be defined as follows (Hinton and Salakhutdinov, 2006): E ϱ s , d = − ∑ i = 1 m a i s i − ∑ j = 1 n b j d j − ∑ i = 1 m ∑ j 1 n d j w j , i s i . (13)

Then, the joint probability distribution of s, d is given as follows: P ϱ s , d = 1 Z ϱ e − E ϱ s , d (14) where Z ϱ = ∑ s , d e − E ϱ ( s , d ) represents the normalization factor. Afterward, the marginal probabilities with which the model assigns to a visible vector s and a hidden vector d are given by P ϱ s = ∑ d P ϱ s , d = 1 Z ϱ ∑ d e − E ϱ s , d (15) P ϱ d = ∑ s P ϱ s , d = 1 Z ϱ ∑ s e − E ϱ s , d . (16)

Combined with the joint probability distribution, marginal probability distribution, and sigmoid function, we can get the probability of activation of a neuron on the hidden layer when given the visible layer state is given as follows: P d τ = 1 | s = s i g m o i d b τ + ∑ i = 1 m w τ , i s i (17) The probability of activation of a neuron on the visible layer when given the hidden layer state is given as follows: P s τ = 1 | d = s i g m o i d a τ + ∑ i = 1 n w j , τ d j (18) where s i g m o i d ( x ) = ( 1 + e − x ) − 1 .

Given training set X = s ¹, s ², … , s ^l , S i = ( s 1 i , s 2 i , … , s m i ) T , where l is the number of training samples. Training the RBM is to adjust the parameters ϱ = (w, a, b) so that the probability distribution represented by the RBM is consistent with the training data as far as possible.

The parameters ϱ = (w, a, b) can be solved by maximizing the likelihood function of visual layer data as follows: P ϱ s = 1 Z ϱ ∑ d exp s T w d + a T d + b T s . (19)

Considering the contrastive divergence (CD) algorithm proposed by Hinton (Hinton, 2002), the conditional probability distribution of the hidden layer and visual layer is given as follows: P d | s = ∏ j P d j | s , P d j = 1 | s = 1 1 + exp ∑ i w i j s i − a j (20) P s | d = ∏ i P s i | d , P s i = 1 | d = 1 1 + exp ∑ j w i j d j − b i . (21)

The updating rules for the parameters are given as follows: Δ w i j = ϵ 〈 s i d j 〉 d a t a − 〈 s i d j 〉 r e c o n (22) Δ a i = ϵ 〈 s i 〉 d a t a − 〈 s i 〉 r e c o n (23) Δ b j = ϵ 〈 d j 〉 d a t a − 〈 d j 〉 r e c o n (24) where Δ is the learning rate; Δw _ij , Δa _i , and Δb _j are the updating values of the weight and biases in the visible layer and hidden layer, respectively; ⟨⋅⟩ _data means the values of visible variable i multiplied by hidden variable j before reconstruction; and ⟨⋅⟩ _recon means the values after reconstruction. ⟨⋅⟩ _data and ⟨⋅⟩ _recon represent the distribution defined by the reconstruction model.

Reconstructed visible variable s′ and hidden variable d′ are one sampling of the original variable p (s, d); therefore, the sample set obtained by multiple sampling can be regarded as an approximation of the original variables.

The typical deep belief network (DBN) structure was proposed by Hinton and Salakhutdinov (2006). It is a deep learning neural network which consists of three RBM layers and one back propagation (BP) layer. The advantage of the DBN is its capability of extracting nonlinear features by hidden layer units through unlabeled training data in visible layer units. Training a DBN contains two steps. The first step is pretraining: weights of the generative model are obtained through unsupervised greedy layer-by-layer pretraining. The second step is fine-tuning, which improves the generalization ability of the discriminant model using the gradient descent algorithm.

According to the contrastive divergence (CD) algorithm, in the pretraining step, the weights and the bias of DBN are updated by w = w + Δ p d 0 = 1 | s 0 s 0 T − p d 1 = 1 | s 1 s 1 T (25) b = b + Δ p d 0 = 1 | s 0 − p d 1 = 1 | s 1 (26) a = a + Δ s 0 − s 1 (27) where p (d ⁽⁰⁾ = 1|s ⁽⁰⁾) = σ(w _j ⋅ s ⁽⁰⁾ + b _j ) is the probability of the neurons in the hidden layer that will be turned on for Bayesian visible neurons, and Δ is the learning rate.

In the fine-tuning step, for the output BP layer, the minimum mean square error (MSE) criterion is used, and the cost function is as follows: E = 1 N ∑ i = 1 N X ̂ i w l , b l − X i 2 (28) where E is the MSE of DBN, X ̂ i and X _i represent the expected output and the actual output, respectively, i is the sample index, and (w ^l , b ^l ) represent the weights and bias parameters to be learned in the lth layer.

The gradient descent method is used to update the weights and bias parameters of the network according to w l , b l = w l , b l + Δ ⋅ ∂ E ∂ w l , b l . (29)

To realize a balance between the local and the global search, a velocity update equation that depends on the mode with Markov parameters is introduced in the PSO algorithm. In general, the particles in the early stage should maintain their independence and diversity, which is helpful to expand the search area and avert premature convergence to local optimum. In the end, all groups may converge to the best particles and obtain a more accurate solution. The evolutions of the particles’ velocity and position are shown as s i τ + 1 = w ξ τ s i τ + c 1 ξ τ γ 1 p 1 τ − x i τ + c 2 ξ τ γ 2 p g τ − x i τ x i τ + 1 = x i τ + s i τ + 1 (30) where w (ξ(τ)), c ₁ (ξ(τ)), and c ₂ (ξ(τ)) are the inertia weight and acceleration coefficients. They all depend on a Markovian chain. The Markov chain ξ(τ) takes values in a finite state space R = 1,2 , … , N with probability transition matrix Π τ = ( π i j ( τ ) ) N × N . π i j τ ≥ 0 ( i , j ∈ R ) stands for the transition probability from i to j satisfying ∑ j = 1 N Π i j τ = 1 . According to the current search information, Π ^τ can be adjusted by D i v k = 1 R ⋅ ∣ L ∣ ∑ i = 1 R ∑ ι = 1 O x i ι τ − x ̄ ι τ 2 (31) so as to balance the global and local search abilities, where R represents the size of the swarm. The length of the longest diagonal in the search space is denoted by ∣L∣. The dimension of the object problem is O. x _iι is the ιth value of the ith particle, and the ιth value of the average point x ̄ is represented by x ̄ ι , which is shown as x ̄ ι = 1 R ∑ i = 1 R x i ι τ (32)

In the study, the BP layer of DBN is replaced by the SPSO-SVM classifier (For detail of switching PSO to optimize the parameters of SVM, refer to the study by Lu et al. (2015).

This section introduces rice disease identification based on the DBN and SPSO-SVM framework. The top layer of the traditional DBN model is the BP neural network used to complete the prediction. In this study, the hybrid kernel support vector machine method optimized by the self-adaptive particle swarm optimization (SPSO) algorithm based on particle swarm concentration is used to replace the BP neural network, Combined with the DBN algorithm to detect rice diseases, the improved DBN model is shown in Figure 6.

This section introduces the process of rice diseases image via the DBN and SPSO-SVM framework. The flowchart of rice diseases image recognition based on the DBN and SPSO-SVM is shown in Figure 7, and our aim is to learn features to identify the rice diseases. To reduce the computational complexity in the process of feature extraction by the DBN, we extract the region of interest (ROI) containing the lesion. The ROI is extracted by combining interactive and automatic extraction methods. First, the experienced plant protection experts mark the outline of the largest section of the lesion in the rice disease images. Then the maximum outer rectangular frame is constructed with the largest section. Finally, the rectangular frame is moved up, down, left, and right by five pixels to obtain the final ROI frame. We adjust the extracted ROI to a uniform scale of 64 × 64 to compare the discriminatory ability of different images against rice disease, and to adjust the network structure and parameter to analyze the impact of classification diagnosis results.

The selection of the input feature is very important for the DBN because it plays a decisive role in the performance of identification. In this study, the color feature, the shape feature, and the texture feature are selected as the input features of the DBN. The color feature is selected because infected rice tissue usually changes its color from green to brown to yellow. The texture feature is selected to detect the rice disease as there are great differences in thickness and direction between healthy and diseased tissues. Also, the shape feature can better capture the edge and internal pixel features of the lesion target.

In the pretraining stage, after three features are input into the visible layer of the first RBM, each layer of the DBN network is trained independently by an unsupervised algorithm, then we obtain the weight of the generated model. In the fine-tuning stage, the SPSO-SVM is proposed to take the output eigenvector of RBM as its input eigenvector and then train the SPSO-SVM classifier using the supervised algorithm. Moreover, in order to ensure the whole DBN eigenvector mapping is optimal, the backpropagation network will also forecast the error information from back to front to each layer, and then the weights of the DBN are fine-tuned. Moreover, for the sake of avoiding falling into the local optimum of the SVM, the SPSO has been introduced.

The images are processed and the algorithm is implemented by using the Python 3.6 and TensorFlow 2.0 + Keras 2.2.4 open-source framework for deep learning with the Ununtu 18.04 X64 operating system. The computer has 64 GB memory and Intel (R) i7 5960K CPU at3.5 GHz processor. NVIDIA Titan XP graphics card is used to improve the image processing speed.

The identification of rice blast, sheath blight, and brown spot based on the DBN and SPSO-SVM mainly refers to the influence on the accuracy. After experimental comparison, it was finally determined that the number of RBM hidden layers was 15, the number of hidden layer nodes was 128, the batch size was 64, the learning rate was 0.01, the optimizer was SGD, and the momentum was 0.9. The comparison of test results is shown in Figure 8.

For the sake of verifying the stability and performance of the DBN and SPSO-SVM models, the 10-fold cross-validation method is applied to validate the recognition results. After the model recognizes the rice diseases 30 times, the hit rate, error rate, accuracy, and the area under the receiver operating characteristic (ROC) curve (AUC) value of the model are counted. Finally, the average values of the four performance evaluation indicators are calculated. The results of cross validation are shown in Table1. Table1 shows that the hit rate and accuracy of the model are 91.37 and 94.03%, respectively, and the error recognition rate is 8.63%.The accuracy comparison curves of the four models are shown in Figure 9. It shows that the model achieves high recognition ability and guarantees a low false detection rate. The significance of the AUC value is that the closer the AUC value is to 1, the better is the recognition performance of the model. The AUC value of the recognition model proposed in this study reaches 0.97, which shows that the model has good recognition performance. The closer the ROC curve of the model is to the Y coordinate axis on the left and the closer to the y = 1 line on the top, the better is the performance of the model. The ROC curve of the model is presented in Figure 10. Figure 10 shows the ROC comparison curves of the four models. Through the comparison, it can be seen that the DBN and SPSO-SVM models used in this study have good recognition performance and are better than the other three models. It can effectively learn the implicit feature information in the data set so as to achieve better recognition effect.

The proposed DBN and SPSO-SVM models were used to identify rice blast, sheath blight, and brown spot. Under the same experimental conditions and the same data set, support vector machine, the SPSO-SVM and convolutional neural network (CNN) models are used for comparison. The parameters of the support vector machine are set as follows: the recognition accuracy under different combinations of SVM penalty parameter C and kernel function parameter g is compared by the grid search method; when the penalty factor C of the support vector machine is 2.8 and the kernel function parameter g is 50, the recognition accuracy is the highest. The SPSO parameters are set as follows: initial population 80, inertia weight 0.9, the maximum number of iterations 300, and learning factor C1 is 1.6 and C2 is 1.8. The parameters of the CNN are set as follows: the batch size is 64 and the number of iterations is 500. Using the cross-entropy loss function, the Adam optimizer is used to minimize the loss function, and the learning rate is set to 0.001. The comparison of the results of the four models is shown in Table2. It can be seen that the DBN and SPSO-SVM models gain better results.