The DAE2ENet reconstructs the decoder based on the LSTM-based E2E structure and combines it with dual attention mechanisms. Figure 2 shows the overall structure of the DAE2ENet model, and Figure 3 shows the structure of LSTM-based E2E as well as the operational structure of the LSTM cell. During the whole information flow of the encoding block, the LSTM transfers the input sequence into vector representation according to forward direction. LSTM is an RNN based on a gating strategy. It can effectively solve information loss caused by gradient vanishing in traditional RNNs.

The calculation process for three gate mechanisms is listed as follows.

Finally, the brief output function calculated by the LSTM unit can be depicted as Equations (9) and (10),

where LSTMCell(·) represents a set of calculation rules for each gate mechanism. Notations θ and θ′ are the set of training parameters, which contain {W_f, W_i, W_C, W_o, b_f, b_i, b_C, b_o}.

The attention mechanism has become a standard paradigm in deep learning to solve information overloading and re-allocating problems in sequential models (see Figure 4). An attention mechanism using a key-value pair is included in DAE2ENet, which contains three components: query, key, and value. The query and key vectors are calculated through dot-product to obtain the basic attention score between each current q_i and different k_i, and the softmax function is used to map this score α_i. The weight α_i and value v_i are calculated through multiplication to obtain the final attention score based on weighted summation. The calculations are given in Equations (11)–(13).

where D represents the dimension of the query vector, s(q_i, k_i) represents the score function for q_i and k_i.

According to Figure 4 and Equation (13), the product of the values vector v_i and weight α_i obtained by Equation (12) is the attention value between the query vector and the key vector.

Based on the basic attention mechanism, we design dual attentions in DAE2ENet as shown in Figure 2. For the encoder, all the hidden states are inputted to calculate attention, which is considered global attention (see Figure 5A). The value of global attention is calculated by inputting the states h′ of the encoder and the decoding state hdecoder′ of the decoder as H. The calculations of A_global are given in Equation (14)

where Hq, Hk, Hv denote the query, key, and value vectors of global attention. For the decoder, multi-head self-attention is adopted to calculate the relationships of the input sequence, which is considered as local attention (see Figure 5B). In multi-head attention calculation, a group of attention vectors Q_τ, K_τ, V_τ can be obtained by input X_t, and the header value of head_τ is calculated as Equation (15)

The calculations A_local of all headers are concatenated as Equation (16),

where W_MH is used for the weight parameter that can be learned during training, and concat(·) refers to the concatenation function, which is used to connect the outputs of multi-header self-attention.

Finally, the predicted value of ŷ_t+1 can be obtain by Equation (17).

where FCNN denotes fully-connected neural networks, and MLP denotes multi-layer perceptron neural networks.

Experimental evaluation is an important component of conducting numerical experiments. During the model training process, we chose mean squared error (MSE) as a means to quantify the disparity between the estimated and observed outcomes, and thereafter adjust the model parameters via the backpropagation method. This helps improve prediction accuracy by minimizing errors compared to the true values, thus achieving the purpose of model training. After the model training is completed, we rely on root mean square error (RMSE) metrics to measure our model output accuracy. Smaller MSE and RMSE values suggest a stronger agreement between the predicted and actual results. The Formulas of MSE and RMSE are given as Equations (18) and (19).

where p indicates sample quantity, ŷ_i denotes predicted positions and y_i represents real navigation positions.

In the experiments, the classic optimization algorithm Adam (Kingma and Ba, 2015) is used to modify the adjustable variables in our model architecture, and the learning rate needs to be determined during operation. The sequence information encoding section of the model is frequently formed by three LSTM network structures. The hidden layer is investigated to find the optimal value within the range of [16, 320]. The magnitude of this parameter indicates the degree of non-linearity for fitting the model. When it is large, the model exhibits overfitting of the training set. For each epoch, we train 5,120 samples, which is repeated for 2,000 times. Additionally, to prevent overfitting of the model, dropout (Srivastava et al., 2014) and regularization terms were employed during the training process. Through numerous experiments, the optimized ranges, interval granularity, and optimal parameter values of the model were determined in Table 3.

In comparison with baseline models, we only use navigation status as input X_nav = {X_nav(t), X_nav(t − 1), …, X_nav(t − m)}, where X_nav(t) = {Lon(t), Lat(t), Spe(t), Cou(t)}. The output is the predicting position of ŷ(t + 1) = {Lon(t + 1), Lat(t + 1)}. The last column of Table 4 shows the optimal parameter values of each model during training. When SVM is used for regression prediction experiments, we chose the Gaussian radial basis (RBF) function as kernel and selected the penalty coefficient c = 2.1 for the objective function and the coefficient gamma = 0.02. For neural network models, the optimal parameter of this column represents the number of hidden layers, number of hidden units, learning rate, and regularization value, respectively. According to the results of Table 4, deep learning methods based on LSTM have achieved better performance compared to traditional methods (such as LR, SVM, and BPNN). On the other hand, models incorporating attention mechanisms performed better than baseline methods. LAE2EDNet and GAE2ENet performed worse than DAE2ENet, which reveals that incorporating the dual-attention mechanism boosts overall performance. In addition, compared with DAE2ENet, the variant model DAE2EMLP also has poor performance, indicating that when using LSTM as the encoding and decoding structure, gated mechanisms can more effectively extract information from sequential data.

The visual representation of the experimental results is depicted in Figure 7. Our model of DAE2ENet predicts a route that is consistent with the actual location of Route 1. Figure 7A displays the forecast outcomes of each model, while Figure 7D highlights discrepancies between actual and estimated movement trajectories. The overall comparison shows that predictions of longitude and latitude obtained by DAE2ENet have the smallest errors compared to other models. Figures 7B, C show more details about the performance of DAE2Net in predicting turning and straight navigation. Especially during ship turning, its prediction error gradually increases in the longitudinal direction due to changes in the navigation status of the ship. However, the prediction result of our model is more stable and has less error compared to other models. Qualitative and numeric analyses provide additional confirmation of the functionality and implementation ability.

Route 2 is used as validation, and the results are shown in Figure 8. Route 2 has larger turning angles, which makes prediction more difficult. As shown in Figure 8D, the prediction errors of BPNN and LSTM have greater volatility and worse prediction performance, especially during navigation turning. DAE2ENet shows lower prediction errors with smaller variations and indicates superior generality and reliability for trajectory prediction.