Network embedding methods [1,5,10,12,13] are used to learn low-dimensional and dense vector representations for nodes in real graph networks, while preserving network structure and facilitating further analysis of graph networks. Various network embedding techniques have been proposed based on deep learning, inspired by word2vec models such as Skip-Gram [29], including DeepWalk [13] and node2vec [10]. DeepWalk [13] first performs a random walk on a network to generate an unbiased random sequence composed of nodes. The neural network (Skip-Gram) is subsequently used to train network node representations by treating nodes as words and node sequences as sentences. Node2vec [10] extends DeepWalk by introducing two parameters (p and q) used to improve the random walk strategy (i.e. BFS and DFS) exploring a more comprehensive graph structure (a biased random walk). Other network embedding models have focused on mining and analysis for specific network structures. For example, LINE [12] is a classic approach that learns node embedding information by preserving both first-order and second-order proximities in the graph. Similarly, SDNE [30] utilizes a semi-supervised deep autoencoder model to capture first-order and second-order proximities. NetMF [31] unifies DeepWalk, LINE, and node2vec into a single matrix factorization framework.

The techniques discussed above focus on mining graph structure, without considering node attribute information. However, nodes in real-world networks often contain rich attribute data, such as abstract text in a publication network and user profiles in social networks (called attributed networks). Considering node attribute information in the learning process has been shown to improve the quality of network representation learning and provide a more comprehensive node embedding strategy to facilitate downstream tasks [5,32–34]. TADW [32] has been used to demonstrate the equivalence between DeepWalk and matrix factorization in attributed network representation learning. It was also the first algorithm used to jointly learn node attributes (textural features) and network structure, which are achieved via matrix factorization. However, TADW only considers second-order and higher-order proximity, leaving out first-order proximity considerations (i.e., homophily properties). HSCA [33] jointly learns homophily data, structural content, and node attributes to develop an effective node representation. MIRand [34] is an unsupervised algorithm for attributed single-layer graph network embedding based on random walk. This approach first establishes a two-layer graph network, one of which depicts structural information from the input graph while the other describes node attributes or content. MIRand performs the random walk according to node informativeness, intelligently traversing between structure and attribute layers.

Inspired by the success of CNNs in computer vision, graph neural networks (GNNs) [35] generalize 2D convolutions from Euclidean images to non-Euclidean graph data, which provides a powerful end-to-end method for learning node representations while addressing graph-related tasks. Graph convolutions are performed by aggregating neighborhood node information, which naturally considers node attributes. Representative work concerning convolution operators applied to graph data is found in Graph Convolutional Network (GCN) [5]. Michael Schlichtkrull et al. first applied a GCN framework to model relational data, focusing on knowledge graph datasets, which is called Relational Graph Convolutional Networks (R-GCNs) [36]. In contrast, GAT [15] models specify different weights for varying neighborhood nodes when performing convolution operations. GAT assumes that different neighboring nodes exhibit different importance levels for the objective node while aggregating neighboring nodes. GraphSAGE [14] is an extension of the GCN framework that uses inductive node embedding. For a complete overview of network embedding techniques, readers are referred to recent studies on the topic [35,37].

Although these techniques have proven to be effective and efficient in various scenarios, they each attempt to process standard single-layer graphs, which implies the graph only consists of one type of relationship as shown in Figure 1A. However, in practical applications, most networks exhibit multiple relationships between nodes. For example, in social networks, the relationship between two users could be friendship, co-worker, or simply advice [38]. Although these diverse relationships can independently form different networks to be analyzed separately, specific interactions and associations exist among them [17,28].

PMNE [19] uses three methods to learn global embedding information for analyzing multiplex networks, including network aggregation, result aggregation, and layer co-analysis which considers interaction information and thus achieves the best overall performance than network aggregation and result aggregation. Ryuta Matsuno et al. proposed MELL [20] for multiplex networks. It first requires embedding vectors, for the same nodes in diverse relationships, to be close to each other in order to share all layer structures. It then introduces a layer vector that can capture each layer’s connectivity for use in differentiating edge probabilities in each relationship. As such, MELL focuses specifically on link prediction tasks. MVE [21] is a novel collaboration framework for multiplex network embedding, which promotes the collaboration of different views and introduces an attention mechanise to learning the weights of different views. Such can obtain robust node embedding results. However, MVE only consideres the network structure (i.e. attributes of nodes are ignored). MNE [11] uses one high-dimensional common embedding and a lower-dimensional additional embedding for each type of relationship, each of which can be learned jointly based on a unified framework. MANE [39] jointly models both connections in each relationship and network interactions from different relationships in a unified framework. Essentially, MANE focuses on processing heterogeneous graphs with different types of nodes and edges. However, node attributes are not considered by the models discussed above.

HAN [25] uses a novel heterogeneous graph neural network with node-level and semantic-level attentions for attributed multiplex networks, generating node embeddings via aggregating features from meta-path based neighbors. This approach attempts to describe heterogeneous graphs generated from meta-paths considering the semantics between nodes. Similarly, mGCN [22] utilizes GCN to learn node representations for each relationship. In order to jointly learn cross-layer interactions, the authors used a weighted average over relation-specific representations to produce generalized descriptions in which weights were calculated based on projection matrices from different networks. Unlike in mGCN, our proposed approach adds regularized consensus constraints and trainable weight parameters constraints among GCN for different dimensional graph networks on the objective function. In fact, the mGCN only learns node embeddings for each dimension, using a weighted average of the embedding results to generate overall node representations without considering the interactions between different dimensional networks. Masha Ghorbani et al. extended the GCN model to form a multi-layer graph embedding called MGCN [23]. This approach utilizes GCN models to learn node representations within relationships. However, MGCN focuses on multi-layer graphs with explicit adjacency links between nodes with different relationship types. The types of nodes found in these relationship networks can vary widely. For instance, one layer could denote an airport network, while another describes a power grid. MGAT [26], an extension of the GAT model, introduces regularization terms for model parameters on the objective function to optimize multiplex network embedding. The primary difference between our method and MGAT is that we extend the GCN model while requiring less memory than GAT for parameter storage [15]. Furthermore, our technique is advantageous because it uses a random walk network embedding technique to learn 10-hop node information [10] as the pre-trained node feature which concatenated with the original node features, and resulting new node features were input into the GCN model, while MGAT only learns 2-hop data. MGAT also fails to consider the similarities of node embeddings between different dimensional graph network.

The difference between single-layer and multiplex graph network embedding: With the advent of the big data where many different links of interconnected objects, it is difficult to model these interacting objects as single-layer graph networks but can naturally model using multiplex graph networks to describe different links. For example, two users could be connected to each other across multiple social platform (e.g. Twitter, Facebook, and LinkedIn). Therefore, each social relationship can be modeled as a graph network. The traditional graph network analysis methods with single-layer network embedding technique can be utilized to analyze these three graph networks separately, which may result in incorrect analysis results. As the single-layer graph network can only describe some or even biased information between nodes. Another way is to transform these three graph network (e.g. Twitter, Facebook, and LinkedIn) as a weighted or an unweighted single-layer graph network. The weight represents the number of link types (i.e. 0, 1, 2, 3) between connected nodes. Then the single-layer network embedding methods can be used on the transformed single-layer graph network. Although it is easy to perform the graph network analysis in this way, the interactive information among different relationships through the same nodes in different links are ignored. The multiplex graph network can model more comprehensively characterize of the complex systems than single-layer graph network. Besides, most of the multiplex graph network embedding methods capture the interactive information and common information among different relationships, and the differences between different relationships. Therefore, the graph analysis results of the multiplex graph network embedding are more accurate than single-layer network embedding.

Attributed Multiplex Graph Network. Given a single-layer attributed network formally denoted as G = {V, E, X}, the vertices V represent n nodes in the graph. The term E is a set of edges representing the presence of a connection or relationship between two nodes, where e i j = v i , v j ∈ E describes the relationship between node v _i and node v _j . In addition, |V| and |E| denote the size of the vertice and edge sets respectively, and X ∈ R ^n×F is a matrix that represents attributes for the n nodes, F represents the dimension of node features, A is an adjacency matrix for the graph G (with a size of V × V ), and A i j ∈ 0,1 denotes connections for unweighted network graphs. The condition A _ij = 1 represents a link between v _i and v _j , otherwise A _ij = 0. In practical applications, A is typically sparse and high-dimensional, especially for very large-scale networks. Attributed multiplex networks with M different relationship types can be represented as G = {V, E ⁽¹⁾, E ⁽²⁾, …, E ^(M), X}, where G ^(r) = {V, E ^(r), X} is a graph of the relation type (or dimension) r and A = A ( 1 ) , A ( 2 ) , … , A ( M ) is a set of adjacency matrices for the graph G. Multiplex network embedding attempts to learn global consensus node representations for each node v _i ∈ V with a d dimensional dense vector, through better collaboration among different dimensions. This suggests the correlations among diverse relation types should be considered for a comprehensive and informative node representation. Low dimensional and consensus vectors can be represented as z _i ∈ Z ∈ R ^n×d for each node v _i ∈ V, where d ≪ V . These notations are summarized in Table 1.

An Overview of the Framework. The overall framework for AMRG is illustrated in Figure 2. It is primarily composed of four components, including 1) a random walk network embedding model used to capture distance neighboring node information as pre-trained node features, 2) dimension specific node embeddings with the GCN model, 3) cross dimension learning, and 4) an attention-based mechanism used to learn node importance in different dimensions for fusing different dimensions adequately.

We use the node2vec random walk network embedding technique to capture distance node feature of each dimensional graph network, and then the averaged node embedding of M node embeddings are concatenated with the original node feature. The resulting is considered as new node features, which are inputted into the GCN model. Then the GCN model can be utilized for the relation-type specific network G ^r , to learn a set of node representations H _r . However, unlike the conventional GCN method [5], a weight was added to the self-connections which is down in the same way in [24]. Large weights (w > 1) indicate the node itself plays a more important role in generating its embedding than its neighboring nodes in the process of aggregating neighbor information. Furthermore, learnable weight parameters of M GCNs are constrained using a regularization term [26]. In addition, we introduce a regularized consistency constraint for each network embedding H _r∈M to capture node similarities from their counterparts. These two constraints from different dimensional graph network can be beneficial [17,19,24] for the downstream node classification, as this can capture more comprehensive information of the multiplex network. Finally, a global consensus node embedding was generated by weighted average different dimensional network embeddings based on the attention mechanism. This obtained embedding is a comprehensive, higher-quality, informative node representation, which can be used for classification and visualization tasks.

The advantage of GCN is that it not only considers the network structure, but also fuses the node attributes. GCN is typically using two convolution layers [5], which means that it can only captures 2-hop node neighboring information. However, more than two convolution layers will result in the over-smoothing problem. Node information from a larger receptive domain (10-hop) with node2vec random walk network embedding technique [10] can help to capture richer node features. However, node2vec leaves out of consideration the node attributes. We thus combine the node2vec random walk network embedding technique and GCN to learn the node embeddings of multiplex graph network. The resulting node embeddings can not only fuse node’s attributes, but also make up for GCN’s inability to learn distance node information to capture the structural equivalence.

Given the rth dimensional network, we can utilize the node2vec random walk technique to learn 10-hop neighboring node information. The resulting network embedding can be represented as X r w r ∈ R n × d r w (d _rw ≪ n). For M different dimensional networks of multiplex graph network, we can use the same method to acquire X r w 1 , X r w 2 , … , X r w r , … , X r w M . Then, we average M node embeddings as: X r w = 1 | M | ⋅ X r w 1 + X r w 2 + ⋯ + X r w r + ⋯ + X r w M (1)

The X _rw is considered as the pre-trained node feature which contains the distance neighboring node information. Then the X _rw is concatenated with the original node attributes X (i.e., X n e w = X + X r w , X n e w ∈ R n × ( F + d r w ) ). X _new , considered as the new node feature, is inputted into the GCN model.

Graph convolutional neural networks provide a powerful solution for generating node representations for a given graph [5], which naturally incorporate node attributes. In this section, we utilize multi-layer GCN to learn the dimension specific node embedding. For a given input graph (A, X), the layer-wise propagation rule can be expressed as [5]: H l + 1 = σ D ̃ − 1 2 A ̃ D ̃ − 1 2 H l W l (2) where A ̃ = A + I n is an adjacency matrix with added self-connections, I _n is the identity matrix,X ∈ R ^n×F is a feature matrix for the input graph, D ̃ i i = ∑ j A ̃ i j is the degree matrix for A ̃ , W ^(l) is the trainable weight parameters matrix for the lth layer in the GCN, σ(⋅) denotes an activation function (i.e., Re LU • = max 0 , • ), and H ^(l) ∈ R ^n×D is the activation matrix for the lth layer, initialized as H ⁽⁰⁾ = X. In this way, the resulting node embeddings that capture node attributes X and the graph structure A simultaneously [5].

For a dimension specific network G ^r , the representation learning model can be denoted as: H r l + 1 = σ D ̃ r − 1 2 A ̃ r D ̃ r − 1 2 H r l W r l (3) where A ^r ∈ R ^|V|×|V| is an adjacency matrix for the graph G ^r and D ^r is the corresponding diagonal degree matrix. Unlike in conventional GCN, we modify A ̃ r and define it as A ̃ r = A r + w I n Park and Kim [24]. In this expression,w ∈ R is the weight of self-connections used to measure the relative importance between objective nodes and its neighboring nodes in generating objective node embeddings. A value of w > 1 implies the objective node itself is more important than its neighboring nodes, with increasing values of w representing higher importance. The term D ̃ i i r = ∑ j A ̃ i j r represents the degree matrix for A ̃ r . In this paper, a two-layer GCN was used to learn dimension specific node embeddings, and the node feature of the input graph is X _new , i.e.H ⁽⁰⁾ = X _new . The last layer output embedding matrix was denoted H _r , which describes a dimension specific node embedding for the graph G ^r . The resulting node embeddings H _r that capture node original attributes X, distance node neighboring information X _rw and the graph structure A simultaneously [5].

For a given dimension r ∈ M, we can obtain the node representation H _r∈M ∈ R ^n×d , which provides distance information for G ^r . Each H _r∈M is acquired independently by training a two-layer GCN model, as described by Eq. 3. However, these embedding matrices fail to take advantage of interactions and similarities between diverse dimensions. This inspired us to devise a way of jointly learning embedding information from diverse dimensional networks to develop a more comprehensive node representation. This task was accomplished by adding two regularization constraint mechanisms to the objective function representing the consistency constraint among the M network embedding matrices H _r∈M , and the trainable weight parameter constraint among M GCNs. These two constraints are discussed below.

Regularized consistency constraints: We first applied the normalization to each node embedding matrix to obtain the normalized matrices (i.e. transforming the H _r to H _r−nor ). The normalized matrices were then exploited to collect similarity information between each pair of counterpart nodes by H r − n o r ⋅ H r − n o r T , and the resulting is the similarity of n nodes. The regularized consistency constraint can be defined as follows: L C C = ‖ H r − n o r ⋅ H r − n o r T − H r ′ − n o r ⋅ H r ′ − n o r T ‖ 2 2 , r ∈ M , r ′ ∈ M , r ≠ r ′ (4) where ⋅ ^T represents the transpose. This constraint can adaptively capture node similarity information among diverse dimensional graph networks when training the proposed model oriented by the node classification task.

Regularized trainable weight parameter constraints among M GCNs: As in Xie et al. [26], we utilize regularized constraints for trainable weight parameters among M GCNs models. This constraint can be defined as follows: L W P = ∑ r = 1 M ∑ r ′ = 1 , r ′ ≠ r M W r − W r ′ 2 2 (5) where W _r and W _r’ are trainable GCN weight matrices for the relation type r and r′.

Now attention mechanisms were used to learn corresponding importance weights from different dimensional graph network during each step of model iteration. When optimization of the proposed model ceases, the learned weights represent the importance of diverse dimensional graph networks. The learned dimensional attention matrices for the n nodes were used to embed H _r∈M into the final global consensus node representation as follows: Z g l o b a l = ∑ r = 1 M α r H r , α r ∈ R n × n , H r ∈ R n × d (6) where α _r is the learned importance of n nodes for the r dimension network.

Now, the node v _i can be used as an example to illustrate how importance values can be acquired for the node v _i and how attention matrices α _r were acquired for the relation type r. For each r ∈ M, the embedding of node v _i in H _r is given by the row vector h r i ∈ R 1 × d . We first transform h r i via a nonlinear transformation (i.e. the f(x) function in Eq. 7) and then apply a shared attention vector p ∈ R ^h ′^×1, designed to determine the weight ε r i : ε r i = p T ⋅ f W ⋅ h r i T + b (7)

Here, W ∈ R ^h ′^×d is a weight matrix, b ∈ R ^h ′^×1 is a bias vector, and f ( x ) = exp ( x ) − exp ( − x ) exp ( x ) + exp ( − x ) is the tanh function (an activation function). The Softmax function can then be used to normalize different attention values in diverse dimensional graph networks. The final weight for node v _i can be calculated as: a r i = exp ε r i ∑ r = 1 M ⁡ exp ε r i (8)

Larger values of a r i imply that corresponding embeddings are more important. For n nodes in the r dimension network, we can first obtain a learned weight column vector a r = [ a r i ] , a r ∈ R n × 1 and then transform the column vector a _r into a α _r = diag (a _r ), α _r ∈ R ^n×n diagonal matrix.

Node Classification: The output embedding matrix in Eq. 6 was used for node classification tasks in combination with a linear transformation and a Softmax function. Prediction results were then calculated as follows: Y ̂ = s o f t m a x W n c ⋅ Z g l o b a l + b n c (9) where W _nc ∈ R ^n×n is a weight matrix, b _nc ∈ R ^n×C is a bias vector, Y ̂ = y ̂ i c ∈ R n × C is the predicted result for n nodes, and y ̂ i c represents the predicted probability of node v _i belonging to class c. The s o f t m a x x = exp ( x ) ∑ c = 1 C ⁡ exp ( x c ) function is a normalizer across all classes. The cross-entropy loss was then minimized over all training nodes using a loss function defined as follows: L n c = ∑ v i ∈ S ∑ i = 1 C − y i ⁡ ln y ̂ i (10) where S represents the training set, y _i is the real label for node v _i , and y ̂ i is the predicted label.

Overall Objective Function: The consistency constraint in Eq. 4 was jointly optimized, along with the trainable weight parameter constraints in Eq. 5 and the node classification function in Eq. 10. An overall objective function was given by: min θ L = min θ L n c + α L C C + β L W P (11)

Here, α and β are hyper-parameters used to control the importance of consistency constraints and trainable weight parameter constraint terms. Labeled data were then used to guide the learnable parameters θ = W r , α r , W , b , W n c , b n c , r = 1,2 , … , M , which were automatically learned and updated via gradient descent and back-propagation algorithms. Convergence of the overall objective function L could then be used to obtain a global node representation Z _global . This process is summarized in Algorithm 1.

Algorithm 1

The proposed technique.

For a given multiplex graph network G = V , E ( 1 ) , E ( 2 ) , … , E ( M ) , X with M relationships, we first use the node2vec random walk technique to capture the 10-hop neighnoring node information, and then Eq. 1 is utilized to obtain the pre-trained node feature X _rw concatenating with the original node attributes X to get X _new . The proposed multiplex graph convolutional network is used to encode all nodes of the multiplex graph network into vectors. We first set the hyper-parameters including the embedding dimension d, parameters α, β, w. When training the proposed model oriented with the node classification task, the learnable parameters are first random initialized, and optimized with the back-propagation algorithm. While the loss function in Eq. 11 has not converged, the proposed model will computer each dimension (relationship) node embedding with Eq. 3, cross dimension interactions with Eq 4 and Eq. 5 and attention coefficient using Eq. 7 and Eq. 8. The global node embedding Z _global for all nodes from the multiplex relationships is generated according to the obtained attention coefficient.

Our proposed technique is primarily composed of four components: 1) the pre-training of node features using a random walk network embedding method, 2) dimension specific node embedding using a GCN model, 3) cross dimension modeling terms, and 4) an attention-based mechanism used to generate global node representations by integrating embeddings of different dimensions. The time complexity of random walk can be expressed as O (|V|). Dimension specific embeddings O (L|E|d′ + L|V|(F + d _rw )d′) could then be learned using a GCN model, where F + d _rw is the dimension of input node features and d′ is the output dimension of one convolution layer. The term L represents the number of GCN layers (2 in this study). The time complexity required for learning global node representations is given by O (|V|d′|M|), where |M| is the number of relation types. Updating attention weights for diverse dimensions in the process of model training, the time complexity can be expressed as O (|S|d′|M|), where |S| is the number of training data. In practice, this quantity of training data is typically small, such that |S| ≪ |E|. In most practical networks, |V| ≪ |E|. As such, the total time complexity of node classification tasks can be simplified as O (|V|(F + d _rw )d′ + |E|d′).

We construct our experiment on the popular Pytorch framework (https://pytorch.org). All the experiments are performed on a computer with 2.6 GHz 4-core Intel Core i9 processor and the GPU is RTX2080.

Datasets: The proposed method was evaluated on several real-world datasets, as described in Table 2. Lazega is a dense network, while the other datasets are sparse networks.

• Citeseer [5]:Citeseer is a citation network consisting of 3,312 research papers, where nodes are publications divided into six different research areas [5]. Node features are bag-of-words representations for individual papers. We can construct the multiplex graph network including two dimensional: a citation dimensional network (where edges represent citation links between papers) and a paper similarity dimensional network. It is a k-nearest neighbor (kNN) graph constructed by calculating the cosine similarity based on the node features and edges representing the top 10 similar papers (i.e., k is 10).

Baseline: AMRG was compared with the following competitive baselines.

• DeepWalk [13]: DeepWalk is designed for standard single-layer network embedding without considering node attributes [13]. This approach first utilizes random walk in the networks and then applies a skip-gram algorithm to learn node representations.

MGAT was implemented using Pytorch, though the source code is not provided here. The source code published by the authors was utilized for all other baselines.

Parameter Settings: The output node embedding dimension for all datasets was set to 32 to provide a fair comparison. We carefully turn parameters of our proposed model to get optimal performance. For our method, a two-layer GCN was trained with hidden layer dimensions of 64, 128, 256,768, 512, 1024 and output dimensions of 32. The objective function in Eq. 11 was minimized for the training set using a learning rate of 0.000 95–0.005 and the Adam optimizer. A dropout rate of 0.5 was used in addition to weight decay values of 0.000 three for Cora, 0.002 for Citeseer, 0.000 seven for Lazega, 0.000 five for DBLP and ACM, 0.000 nine for Amazon and 0.03 for IMDB. Consistency constraint coefficients and trainable weight parameters (α and β) were searched in the intervals 0.001 , 0.05 , 0.2 , 0.9 , 1.0 , 0.1 and 0.04 , 0.05 , 0.1 , 0.5 , 0.6 , 0.7 , 1.0 , respectively. The self-connection weight was set to 2.0 for Citeseer and Amazon datasets, 3.0 for ACM and IMDB datasets and to 1.0 for Cora, Lazega and DBLP datasets. The Lazega dataset contains node relationship types but not node features. As such, a unit diagonal matrix was used as the feature matrix. The node2vec random walk node embedding dimension (d _rw ) for learning distance node information was set to 8. These values of parameters are summarized in Table 3.

For baselines, DeepWalk is a special case of node2vec with p = q = 1. In the conventional node2vec algorithm, hyperparameters were set to p = 2 and q = 0.5, with a window size of 10 and five for negative samples. Other baseline hyperparameters were set as in the original papers.

Accuracy (ACC) was used to evaluate node classification performance for the all datasets. We randomly selected 10% of the nodes to establish a training set, 10% to form the validation set, and the remaining 80% formed the test set. A total of 120 epochs were used for each of the three datasets. The proposed approach was compared with its variants and state-of-the-art baselines to monitor node classification performance for the following scenarios.

• AMRG/rw: A random walk strategy was not used to learn distance neighboring node information.

The baselines of Node2vec and NetMF were designed for standard single-layer networks. As such, we first acquired node embeddings for each dimension separately and averaged the resulting embeddings together to produce the overall node embeddings. This process was repeated 5 times to produce averaged results for baselines. For our proposed method, as the node2vec random walk technique to capture the 10-hop distance node information and the GCN model will produce a slightly different outcome each time, so the performance of the node classification for our proposed model will be evaluated 5 times, and average 5 times results.

Node classification accuracy (ACC) values are reported in Table 4, where the bolded number denotes the best result. The following are evident from the table:

• Compared with all other baselines, our proposed method consistently achieved the best performance for all datasets. These results demonstrate the effectiveness of our model for attributed multiplex network embedding, oriented by node classification tasks. From Table 4, we observe the random walk technique improved the accuracy of node classification by learning 10-hop node information. This suggests that node information from larger receptive fields is important.

Similarly, node classification F1-score values also are presented in Table 5, where the bolded number denotes the best result. These results demonstrate that our method is stable and competitive.

Now we use the Lazega,a dense network, and three sparse networks (Cora, IMDB, and DBLP) as examples to were used to analyze changing trends in attention values for node classification tasks. The results are shown in Figure 3, where the x-axis denotes different numbers of epochs during the model training process and the y-axis is the corresponding attention value. As seen in Figure 3A, the attention values for advice, friendship, and strong coworker dimensional networks are nearly the same for the Lazega dataset in epoch 0. However, this attention value varies with increasing training epochs. The attention value for the advice dimensional network increases in epoch 20 (compared to epoch 0), while the attention value for the strong coworkers dimensional network decreases. Similarly, the attention values for the advice and strong coworker dimensional networks gradually decrease until the model converges. However, the attention value continues increasing for the friendship dimensional network during the training process. This illustrates that our approach can adaptively learn the importance weights of diverse relation type network embeddings during individual steps. For example, information provided by the friendship dimension network was more important than that of the advice and strong coworker dimension for the Lazega dataset. Similar trends were observed for the Cora, IMDB and DBLP datasets. The citation dimension network also proved to be more important than the paper similarity dimension network in the overall system, as shown in Figure 3B.

For the IMDB dataset, when the epoch is 18, the accuracy of node classification is the best (i.e. 70.28). At this moment, the attention value for co-actor network and co-director network is almost equal. From Table 4, we can see that when the attention value for co-actor network and co-director network is equal (i.e.AMRG/att, attention values are 0.5 and 0.5 respectively), the accuracy is 70.00, which is in close proximity to 70.28. This demonstrate that the attention mechanism is important. For DBLP, when the epoch is 103, the accuracy of node classification is the best.

In this section, we analyze the effectiveness and sensitivity of pre-trained node vector dimensions, two regularization constraints, and the weight of self-connections. Specifically, we evaluate the performance (i.e., accuracy and macro-F1) of our method for node classification tasks with respect to α,β,w and d _rw . Since these datasets produced similar results, we use the Citeseer, ACM and Cora datasets as examples respectively.

Analysis for α: In this test, the value of β was set to zero, which eliminates L _WP in Eq. 11. The parameter α was varied from 0.1 to 1.0, as shown in Figure 4A. The accuracy and macro-F1 score for these node classification tasks remained relatively stable with increasing α for Citeseer. And when the α is 0.3, the accuracy and macro-F1 score is the best for ACM in Figure 4D.

Analysis for β: Here, the value of α was set to zero, which eliminates the consistency constraint L _CC in Eq. 11. Figure 4B demonstrates how different values of the coefficient β affected node classification performance. The value of β was increased from 0.1 to 1.0, as the accuracy slowly increased and decreased (the maximum was in 0.7), and the macro-F1 score remained nearly constant. Similarity trends can be found in Figure 4E and the maximum was β in 0.6.

Analysis for w: The impact of the weight on these self-connections was observed by varying w from 0.0 to 10.0, as shown in Figure 4C for Citeseer. A value of w = 0.0 implies the node itself only considers neighboring nodes in generating embeddings, producing low accuracy and macro-F1 scores. Larger values (w > 1) indicate the node itself is more important than its neighboring nodes. The accuracy first increases and then remains relatively stable, before dropping quickly for w = 10.0. The maximum accuracy and macro-F1 scores occurred for w = 2 and w = 1, respectively, with macro-F1 decreasing slowly as w increased. For ACM in Figure 4F, the best accuracy and macro-F1 occurred for w = 3.

Analysis of random walk dimension d _rw : Figure 5 suggests the accuracy of node classification was maximized for d _rw in 8, decreasing for higher dimensions. This demonstrates that pre-trained node features, which contains distance node neighboring information, can improve model performance. When the embedding dimension d _rw is small, the obtained pre-trained node features are not enough to describe the distance neighborhood information. On the contrary, if the value of embedding dimension d _rw is large, it will introduce some noise on the obtained pre-trained node features.

Analysis of different training nodes: We tested the performance of our method and its variants using training node fractions of 10, 20, and 40% for the attributed multiplex Citeeer network, as shown in Figure 6. The proposed method consistently outperformed its variants, indicating the importance and effectiveness of attention mechanisms, consistency constraints, and trainable weight parameter constraints of |M| GCN for each dimensional graph network for boosting overall performance. The influence of attention mechanisms and these two constraints on node classification performance varied with different training label node ratios.

In the previous section, we analyse the effect of the different single variant on the performance of the proposed model. At present, we take the Citeseer, ACM and Amazon as examples to evaluate the key factor of regularized consistency constraints, weight constraints and attention mechanism. That is to say, which one of three factors is the most significant to improve the AMRG. The evaluation indicator is the node classification accuracy.

• AMRG(wp): The AMRG only with the weight constraints.

Table 6 shows that these three strategies (i.e. regularized consistency constraints, weight constraints and attention mechanism) have different importance on the three datasets. For Citeseer, it shows that the weight constraints is the key factor, as the performance of AMRG (wp) is better than AMRG (att) and AMRG (cc). From the results of ACM and Amazon, we can see that the regularized consistency constraints is the key factor for ACM and the attention mechanism is the key factor for Amazon. As the performance of AMRG (cc) is better and AMRG (att) is better than the other variants.

Task visualization will be performed to provide a more intuitive comparison and to further show the effectiveness of our proposed method. As the results on different datasets will exhibit similar trends, we take the Citeseer dataset as an example to evaluate our method. Output embeddings in the last layer, prior to the Softmax operation, were utilized for node classification tasks and to plot the resulting node embeddings using t-SNE Maaten and van der Hinton [40]. Results for the Citeseer dataset are shown in Figure 7 and are colored using real labels.

It is evident from the figure that the results of the node2vec and NetMF baselines are not satisfactory because nodes with different classes are mixed together. In contrast, MGAT and our method consider node features, producing better results than node2vec and NetMF. This further demonstrates the importance of node features for mining hidden graph information. The three variants are inferior to our method, which is consistent with the results of Table 4. Our approach has the clearest distinct boundaries among the diverse classes and the same classes are grouped together. This demonstrates the importance of pre-trained mechanisms with random walk network embedding, consistency constraints, trainable weight parameter constraints among M GCN, and attention mechanisms.