MicroRNAs (miRNAs) are a class of small non-coding RNAs widely distributed in eukaryotic cells, typically around 22 nucleotides in length. They mainly regulate gene expression through the post-transcriptional processes Hombach and Kretz (2016); Holley and Topkara (2011). In organisms, miRNAs bind to specific target sites on mRNAs, causing their subsequent degradation or translational inhibition, thereby modulating key fundamental physiological processes like cell proliferation, differentiation, apoptosis, and immune system activation Bartel (2009). Recent studies have shown that miRNAs have indispensable functions in a variety of human diseases, including cancer, neurodegenerative disorders, and cardiovascular diseases Li et al. (2023); Dugger and Dickson (2017). They also show great potential in drug response prediction, resistance mechanisms, and therapeutic target discovery Miska (2007); Small and Olson (2011). In this context, studying the subcellular localization of miRNAs is of great significance for understanding their regulatory networks and functional mechanisms Kabekkodu et al. (2018); Catalanotto et al. (2016); Gurtan and Sharp (2013). Different subcellular localizations often suggest that miRNAs are involved in distinct biological processes. Accurate localization prediction not only facilitates the understanding of functional diversification of miRNAs but also provides theoretical support for early disease diagnosis and targeted therapy Jie et al. (2021). Although conventional experimental methods, such as fluorescence in situ hybridization and subcellular fractionation combined with high-throughput sequencing, can directly determine miRNA distributions, these techniques are often complex, expensive, and lack scalability for large-scale samples Thomson and Dinger (2016). Therefore, developing computational methods to efficiently predict miRNA subcellular localization has become a focal point in bioinformatics research. Currently, researchers have developed various machine learning models based on sequence information to explore potential miRNA localization patterns. For example, Huang et al. (2007) proposed a prediction framework combining k-mer frequency patterns with a Support Vector Machine (SVM) classifier, showing the feasibility of using sequence information for localization recognition. However, due to the short length and complex structure of miRNAs, as well as their heterogeneity across different tissues or disease states, models relying solely on sequence-level features often fail to capture the complete biological semantics, resulting in limited accuracy and generalization Li et al. (2014). To address this, some studies have incorporated biological network information, such as the miRNA-mRNA interaction network Hsu et al. (2011), the miRNA-disease association network Jiang et al. (2010), and the miRNA-drug association network Chen H. et al. (2019), to improve prediction accuracy. For instance, Xie et al. constructed a miRNA-target gene interaction network using Graph Convolutional Networks (GCNs) and applied deep learning to predict miRNA functions within cells Guan et al. (2022). Li et al. integrated miRNA, disease, and drug information through a heterogeneous network and used graph embedding techniques for feature learning Sun et al. (2020). Over the past few years, deep learning innovations have significantly advanced bioinformatics research. Convolutional Neural Networks (CNNs) have been used to retrieve sequence features of miRNAs—such as in the DeepMirTar model, which utilized CNNs to improve target gene prediction accuracy Wen et al. (2018). Recurrent Neural Networks (RNNs) have also been employed to capture sequential dependencies, as seen in MirLocNet, which uses Long Short-Term Memory (LSTM) networks to computationally infer miRNA subcellular localization Chen Q. et al. (2019). Graph Neural Networks (GNNs) are widely applied in modeling biological networks. As proposed by Gao et al. (2022), a novel Graph Attention Networks (GATs) combined with biological network information to improve miRNA function prediction Zhao et al. (2022).

In the field of miRNA subcellular localization, researchers have developed various models to enhance prediction accuracy and biological interpretability. MiRLoc Xu et al. (2022), for instance, inferred miRNA spatial distribution by leveraging known mRNA localization and their interaction with miRNAs, reflecting their role in post-transcriptional regulation. MirLocPredictor Asim et al. (2020) incorporated CNNs and positional encoding of k-mers to enhance sequence representation for multi-label localization tasks. DAmirLocGNet Bai et al. (2023a) integrated Graph Convolutional Networks and autoencoders to jointly model miRNA sequence features, disease associations, and disease semantic networks, learning high-level representations from complex graph structures. Some existing excellent models provide us with references. For example, Wang X.-F. et al. (2024) proposed a multi-channel graph neural network framework that integrates multimodal similarity information with hypergraph contrastive learning, effectively identifying novel cancer biomarkers. Wang X.- et al. (2024) designed a directed graph neural network-based multi-view learning model capable of systematically extracting regulatory feature signals from multiple biological layers, enhancing the model’s representational power. Additionally, Wang et al. (2022) developed KGDCMI, a method that integrates multi-source biological information with deep learning techniques to accurately predict interactions between circRNA and miRNA. Comparatively, PMiSLocMF Chen et al. (2024) fused heterogeneous data such as miRNA-mRNA, miRNA-drug, and miRNA-disease networks using a graph attention mechanism, achieving robust performance even in scenarios with sparse data or incomplete labels. Despite improvements, present architectures still face challenges such as inadequate information integration and underutilization of multi-head feature relationships. Effectively integrating multi-source information and building more expressive feature representations to improve miRNA subcellular localization prediction remains an urgent and critical problem.

To overcome these limitations, this study proposes a novel miRNA subcellular localization prediction model named GTMALoc, based on graph transformer and multi-head attention mechanisms. This approach effectively incorporates miRNA sequence information and their roles across different biological networks to improve prediction performance. Specifically, we first extract miRNA features from multiple biological networks–including miRNA sequence similarity, miRNA-mRNA associations, miRNA-disease associations, and miRNA-drug associations—using node2vec. Then, a graph transformer framework is applied to infer latent node correlations, offering better insight into miRNA functionality in different contexts. A multi-head attention mechanism is subsequently employed to integrate miRNA features across networks, capturing deeper, multi-head relational patterns and enhancing predictive performance. The evaluations show that our model outperforms mainstream methods in terms of accuracy, generalization, and stability on public datasets, demonstrating its effectiveness and feasibility in the miRNA subcellular localization task. Key improvements over existing methods provided by this study are:

(1) We propose a new miRNA subcellular localization prediction model that leverages graph transformer and multi-head attention mechanisms to integrate multi-source biological network information.

Network modeling has emerged as a pivotal paradigm in biomedical research due to its intuitive representation of complex relationships, particularly in systematic miRNA analysis involving multimodal correlations. This study integrates four critical biological networks: the miRNA sequence similarity network (quantifying functional conservation), the miRNA–disease association network (revealing pathological regulation), the miRNA–drug interaction network (reflecting therapeutic targeting), and the miRNA–mRNA regulatory network (decoding genetic circuitry). To effectively capture topological features from these non-Euclidean spatial data, we employ the node2vec algorithm Grover and Leskovec (2016), a graph embedding approach based on adaptive random walk strategies. By tuning search parameters—the return parameter p controlling local neighborhood sampling and the in-out parameter q governing global structural exploration—this approach generates semantically preserved node sequences, subsequently vectorized through Skip-Gram modeling. Notably, we implement dimension-specific embedding strategies tailored to distinct network characteristics: 64-dimensional representations in sequence similarity networks to resolve fine-grained patterns of conserved functional motifs, versus 128-dimensional high-capacity embeddings in the three heterogeneous association networks to capture complex multi-hop interactions. This hierarchical embedding mechanism simultaneously reduces feature redundancy while preserving network-specific information, establishing an interpretable mathematical foundation for subsequent multi-view feature fusion.

This study proposes a structure-aware graph neural network, the graph transformer, to learn high-quality node embeddings from graph structures. Unlike traditional GNNs, which struggle with sparse or heterogeneous structures, the graph transformer incorporates multi-head attention and a structure reconstruction loss, enabling better modeling of local and global graph information. First, the input miRNA functional similarity matrix and association matrix are feature fused. After generating the node feature matrix h k at layer k , it dynamically rebuilds the attention weights matrix a k + 1 through dot products of h k and h k T , enabling the joint evolution of topology and features. This gradient-preserving update critically suppresses false-positive edges caused by experimental noise. The model adopts Pre-Layer Normalization (Pre-LN), normalizing features before multi-head attention and feed-forward operations rather than after, which better accommodates high-dimensional biological feature propagation and curbs gradient vanishing in deep training. For extremely sparse data like miRNA-drug networks, binary attention masking embedded in multi-head layers automatically blocks unobserved associations (e.g., unknown miRNA-drug interactions), reducing computational complexity from O ( N 2 ) to O ( E ) (where E denotes valid edges) with less GPU memory consumption, while preventing noise from distorting attention weights. These improvements jointly optimize the model, and the output layer further sharpens the precision-recall curve through dot product similarity and L2 normalized feature constraints in the range of [0,1]. During embedding learning, the model stacks L layers of graph attention modules, where each node’s representation is updated by aggregating the representations of its neighbors (Equation 7): h i ( l ) =   σ   ∑ j ∈ N ( i ) A i j ( l ) W ( l ) h j ( l − 1 ) . (7) Here, σ is an activation function, and W ( l ) is a learnable weight matrix. The attention weights a i j ( l ) are calculated as (Equation 8): a i j l = exp L e a k y R e L U a ⊤ W l h i l − 1 ‖ W l h j l − 1 ∑ k ∈ N i ⁡ exp L e a k y R e L U a ⊤ W l h i l − 1 ‖ W l h k l − 1 , (8) where a is the attention weight vector, which ‖ represents the vector splicing operation. Graph transformer further introduces a multi-head attention mechanism, using K attention heads in parallel in each layer, and finally integrating their output splicing or averaging into the input of the next layer (Equation 9): h i ( l ) = 1 K ∑ k = 1 K σ ∑ j ∈ N ( i ) a i j ( l , k ) W ( l , k ) h j ( l − 1 ) . (9) In order to enhance the model’s ability to express structural information, graph transformer also introduces structural reconstruction loss as the training goal. In the unsupervised setting, the model scores the node pairs of the real edges in the input graph, and defines the structural reconstruction similarity as (Equation 10): s i j = σ h i ⊤ h j , (10) where h i , h j are the final embeddings of nodes i and j, and   σ denotes the sigmoid function. The structural reconstruction loss is formulated as (Equation 11): L reconstruct = − ∑ i , j ∈ E + log σ h i ⊤ h j − ∑ i , j ∈ E − log 1 − σ h i ⊤ h j , (11) where E + represents the set of true edges, and E − denotes negative-sampled non-edges to prevent degenerate solutions where all nodes become indistinguishable. This objective effectively preserves high separability of node connectivity patterns in the embedding space. The final node embeddings H = [ h 1 , h 2 , . . . , h N ] undergo L2-normalization for downstream tasks.

To capture cross-modal dependencies and interactions among various biological features, a Multi-Head Attention (MHA) module is introduced as the core feature interaction component in the fusion model. Based on the transformer encoder, MHA computes attention across multiple subspaces in parallel to enhance local and global correlation modeling. Four input feature types—miRNA sequence, drug features, mRNA features, and disease features are first projected to a common 128-dimensional space using fully connected layers with L2 regularization. These are concatenated and reshaped into a 2D sequence format before being passed into the MHA module. The Multi-Head Attention Mechanism is calculated as follows (Equations 12, 13): Q = H W Q , K = H W K , V = H W V , (12) Attention Q , K , V = softmax Q K ⊤ d k V . (13) The outputs of all heads are concatenated and transformed, the formula is as follows (Equations 14, 15): M H A H = C o n c a t h e a d 1 , … , h e a d h W 0 , (14) H 1 = L a y e r N o r m H + D r o p o u t M H A H . (15)

To further improve the representation capability, the multi-head attention output will be delivered through two layers of Feed-Forward Network (FFN), the formula is as follows (Equations 16, 17): F F N H = R e L U H W 1 + b 1 W 2 + b 2 , (16) H fusion = L a y e r N o r m H 1 + D r o p o u t F F N H 1 . (17)

The final output represents the fused multimodal semantic embedding features, which are used as the input of the subsequent self-supervised learning projection head and the multi-label classification header, which not only retains the information of the original modal features, but also integrates the high-order correlation between them.

During forward propagation, the fused high-dimensional features are processed through the MHA and FFN modules. Finally, the classification head maps the features to predicted subcellular localization probabilities (Equation 18): y ̂ = o f cls x , (18) where f cls is a linear mapping and   o   is the Sigmoid activation function that converts the output into a probability vector over [0, 1]. For each class, if the predicted probability exceeds the threshold (0.5), the class is labeled as positive; otherwise, negative–resulting in the final binary classification output.

To further demonstrate the practical utility of GTMALoc in predicting miRNA subcellular localization, we conduct case studies across seven subcellular categories: cytoplasm, exosomes, nucleolus, nucleus, extracellular vesicles, microvesicles, and mitochondrion. For each compartment, we select the top five miRNAs with the highest predicted probabilities generated by GTMALoc. We then manually verify these predictions against experimental evidence reported in the scientific literature. In total, 35 miRNA–localization associations are examined. As shown in Table 5 of these are supported by published studies, while only five lack current experimental validation.

Taking miR-122 and miR-21 as representative examples, we analyze the alignment between GTMALoc’s predictions and reported biological findings. miR-122 is a liver-specific miRNA that is highly enriched in hepatocytes, and its cytoplasmic localization is well supported by experimental evidence Zhang et al. (2021). Previous studies indicate that miR-122 plays a crucial role in liver homeostasis by regulating lipid metabolism, cholesterol biosynthesis, and HCV replication through mRNA binding Ren et al. (2008). GTMALoc assigns a high confidence score of 0.98 for its cytoplasmic localization and captures its interactions with liver metabolism-related mRNA nodes, highlighting the model’s capacity to extract biologically meaningful features from the molecular network. In contrast, miR-21 is known for its multi-localization behavior and is highly expressed in various cancer types. It has been shown to be secreted via exosomes, contributing to immune modulation and tumor microenvironment remodeling Krishnamurthy et al. (2018), and also localizes in the nucleolus, where it may influence non-coding RNA processing Beckett et al. (2015). GTMALoc successfully predicts both localizations with high confidence and focuses attention on miR-21’s connections to tumor-associated signaling pathways, consistent with its known roles in cell proliferation, anti-apoptosis, and inflammatory response. These case studies suggest that GTMALoc not only achieves accurate subcellular localization predictions but also provides biologically interpretable outputs, particularly for multi-localized miRNAs, offering valuable insights for downstream functional analysis and subcellular mechanism exploration.

In this study, we propose a computational model, GTMALoc, for predicting miRNA subcellular localization. GTMALoc combines graph transformers with a multi-head attention mechanism to fuse heterogeneous biological information from multiple sources. Specifically, the model effectively integrates miRNA sequence features, interaction network structures, and functional properties. Through graph-based modeling and dynamic attention weighting, GTMALoc learns more discriminative high-dimensional feature representations, significantly improving the accuracy of localization prediction. We conduct a comprehensive evaluation of GTMALoc on public datasets. The results show that GTMALoc outperforms existing methods on multiple performance metrics, especially in handling sparse graph structures and high-dimensional feature spaces. Ablation studies confirm the key contributions of each feature modality and attention component to the model’s overall performance. Additionally, through representative case studies, we validate the biological interpretability of GTMALoc. The predicted subcellular localizations are not only consistent with known miRNA functions reported in the literature but also reveal potential regulatory modes that have not been fully explored. Given that miRNA localization is closely related to its regulatory roles in various cellular contexts, accurate localization prediction provides valuable insights into miRNA-mediated mechanisms under physiological and pathological conditions.

Although GTMALoc performs well in various experiments, it still faces some limitations. We integrate multiple heterogeneous features, such as sequence data, functional similarity, and molecular interaction networks; however, biological data often contain noise and incompleteness. For example, the functional annotations of many miRNAs remain incomplete, and some interaction networks may contain missing data or experimental biases, which can adversely affect the accuracy of feature learning. Additionally, differences among data sources in species, experimental conditions, or time points introduce biases and impair the model’s generalizability. In future work, we plan to further refine the model architecture to improve its interpretability, adaptability, and applicability in real-world biomedical research scenarios.