Recent efforts to reduce the computational burden of Vision Transformers (ViTs) have focused on architectural modifications, such as token pruning (Zheng W. et al., 2024) and hybrid CNN-Transformer designs (Chakrabarty et al., 2024). For instance, TrIncNet (Gole et al., 2023) introduced a multi-granularity feature extraction module to identify tomato diseases, while PMVT (Li et al., 2023) adapted MobileViT for real-time plant disease detection. These methods primarily target intra-species tasks, where the model is trained and tested on the same plant species. Nevertheless, they frequently keep all attention heads without acknowledging the redundancy present in cross-species contexts.

A prominent deviation is the research on single-head self-attention in grape leaf disease identification (Zhang et al., 2025), showing a simplified attention approach can preserve accuracy for particular diseases. Nevertheless, this approach sacrifices the multi-head diversity crucial for generalizing across diverse phenotypic traits (e.g., leaf shape vs. lesion texture). TAHA addresses this limitation by selectively pruning heads based on transferability rather than eliminating multi-head attention entirely, retaining diversity while reducing redundancy.

Transfer learning has been extensively employed to address data limitations in plant phenotyping, especially for less-researched species. Early approaches relied on CNNs pretrained on ImageNet, fine-tuning them for target species (Yan et al., 2021). However, these methods struggle with domain shifts caused by interspecies morphological differences, such as leaf venation patterns or canopy structures.

More recent work has explored domain adaptation techniques tailored to biological data. FloraBERT (Levy et al., 2022) adopted attention mechanisms to align gene expression patterns across species, whereas DeepTL-Ubi (Liu et al., 2021) applied transfer learning for ubiquitination site prediction. These methods highlight the potential of attention-based alignment but do not optimize the attention mechanism itself for cross-species efficiency—retaining full computational overhead even when heads are irrelevant to target species. TAHA fills this gap by pruning low-transferability heads, reducing FLOPs by 40.9% while preserving alignment quality.

A parallel line of research has investigated feature space alignment for plant communities (Hirn et al., 2024) and bioacoustic species classification (Zhong et al., 2020). These studies highlight the necessity of retaining discriminative features in domain adaptation, an idea we articulate with our Phenotypic Consistency Constraint (PCC).

Investigating attention heads in large language models has uncovered specialized functions, including positional encoding and syntactic parsing (Zheng Z. et al., 2024). Comparable specialization probably happens in visual processing, where specific heads might identify features common across species, such as the outlines of leaves or patterns in texture.

Pruning methods have been proposed to eliminate redundant heads, either statically (Michel et al., 2019) or dynamically during training (Voita et al., 2019). Nevertheless, these methods usually assess the importance of heads only in one specific area, failing to account for their usefulness across different domains. For example, a head specialized in Arabidopsis leaf shape may be redundant for maize but critical for rice—static pruning cannot adapt to such cross-species differences. Our Head Transferability Score (HTS) extends these ideas by evaluating heads through both domain alignment and phenotypic prediction metrics, ensuring pruning decisions are transferability-aware rather than domain-specific.

The proposed method distinguishes itself by unifying these research threads. In contrast to (Gole et al., 2023) or (Zhang et al., 2025), our approach focuses on optimizing attention heads specifically for cross-species transfer rather than efficiency within a single species. Compared to (Levy et al., 2022) or (Liu et al., 2021), we address computational efficiency through principled head pruning rather than solely focusing on accuracy. Our Domain Alignment Loss (DAL) and Phenotypic Consistency Constraint (PCC) deliver a more refined strategy for retaining features compared to conventional domain adaptation methods (Yan et al., 2021), whereas our iterative pruning framework grants increased adaptability relative to fixed architectures (Zheng W. et al., 2024). This pairing yields a streamlined yet broadly applicable approach for assessing plant traits in various species.

The Multi-Head Self-Attention (MHSA) mechanism operates on input features by employing numerous parallel attention heads, each capturing different dimensions of the visual information. Given an input feature map X∈ℝn×d where n is the number of patches and d the embedding dimension, each head h computes (Equation 1):

where Qh=XWhQ, Kh=XWhK, and Vh=XWhV are learned linear projections for queries, keys, and values respectively, with WhQ,WhK,WhV∈ℝd×dk. All head outputs are joined together and transformed via projection. As shown in Equation 2.

where WO∈ℝHdv×d and H is the number of heads. For botanical imagery, this approach acquires both fine-scale features (e.g., leaf venation) and overarching organization (e.g., plant morphology), though with notable repetition among attention heads (Michel et al., 2019).

Domain adaptation for plant imagery must address three unique challenges absent in general computer vision: (1) interspecies morphological variations (e.g., monocot vs. dicot leaf patterns), (2) phenotypic plasticity under environmental conditions, and (3) limited labeled data for target species. Conventional domain adaptation approaches such as Maximum Mean Discrepancy (MMD) (Gretton et al., 2012) or adversarial training (Ganin and Lempitsky, 2015) frequently do not retain phenotype traits unique to species when performing alignment.

The domain shift between source species S and target species T can be quantified through the discrepancy in their feature distributions, as shown in Equation 3:

where ℱ is a class of functions (e.g., attention heads). For plant phenotyping, we must minimize D(S,T) while preserving the discriminative power for phenotypic traits—a dual objective not addressed by standard domain adaptation approaches (Ganin et al., 2016).

Empirical research indicates attention heads in plant vision models adopt distinct functions, for instance:

This specialization indicates merely a portion of heads might be crucial for cross-species transfer, whereas the remainder contain species-specific traits impeding generalization. Our approach builds on this insight by transferring and pruning specific heads selectively, differing from previous methods which apply uniform treatment to all heads during adaptation (Li et al., 2018).

The alignment process begins with a pre-trained Transformer model on the source species, where each attention head has learned specific feature extraction patterns. For a given input image x, the attention map of head h in the source model is denoted as Ahs(x)∈ℝP×P, where P represents the number of patches. When processing target species data, we compute the corresponding attention map Aht(x) using the same head parameters.

The Domain Alignment Loss (DAL) measures the discrepancy between source and target attention patterns, as shown in Equation 4:

where N is the batch size and KL denotes the Kullback-Leibler divergence. At the same time, the Phenotypic Consistency Constraint (PCC) guarantees the transferred heads keep their ability to distinguish traits of the target species. As shown in Equation 5.

Here, Zi=[Z1(xi),…,ZH(xi)] represents the concatenated outputs of all attention heads, and fMLP is a multilayer perceptron for phenotypic prediction. For datasets with differing phenotypic traits (e.g., PlantVillage focuses on disease classes, CropDeep on growth traits), PCC is computed on shared morphological traits (e.g., leaf shape, venation density) after feature standardization, ensuring consistency across trait types. The correlation score in HTS uses Pearson correlation between head outputs and standardized target labels, mitigating the impact of trait differences.

Each head’s transferability is measured by merging its alignment accuracy and phenotypic importance, as shown in Equation 6.

where α∈[0,1] balances the two criteria, and Corr(·) computes the Pearson correlation between head outputs and target labels. The exponential term transforms the alignment loss into a positive score, with better-aligned heads receiving higher values.

Heads are pruned based on their HTS relative to a dynamic threshold τ, as shown in Equation 7:

where μ and σ denote the mean and standard deviation of all heads’ HTS values, and β controls the pruning aggressiveness. The remaining heads form a compact set ℋ′={h|HTSh≥τ}, reducing the computational complexity from O(HP2d) to O(|ℋ′|P2d).

The trimmed attention approach generates results by exclusively employing the preserved heads, as shown in Equation 8.

where Wo′∈ℝ|ℋ′|d×d is a reduced projection matrix. This approach preserves the initial feature dimension (d) while reducing the quantity of attention operations.

The complete adaptation process follows an iterative procedure:

This cycle repeats until convergence, with λ gradually increased to prioritize phenotypic prediction accuracy in later stages. The step-by-step method supports gradual adjustment while retaining transferable attributes without abrupt loss.TAHA operates under semi-supervised domain adaptation: it requires 10–15% labeled target data to compute PCC and HTS, rather than fully unsupervised. This balances label efficiency and alignment quality, critical for plant phenotyping where labeled data is often scarce.

Table 1 summarizes the datasets used in this study, including their species coverage, task types, and sample sizes. All datasets are used for classification tasks (disease type, growth stage, or phenotypic trait category), as plant phenotyping in this work focuses on categorical attribute prediction.

We compared against four state-of-the-art approaches:

Implementation Details: All models were implemented in PyTorch and trained on NVIDIA V100 GPUs. Our model architecture employed ViT-Base (12 attention heads) as the core component, processing input images scaled to 224×224 resolution. The hyperparameters were set as: α=0.6 in Equation 6, β=1.5 in Equation 7, and λ initialized to 0.1 with linear warmup. The training procedure employed the Adam optimizer with a learning rate of 3e-5 and a batch size of 32. Edge deployment testing: Raspberry Pi 4 (4GB RAM, Cortex-A72), NVIDIA Jetson Nano (4GB RAM, Maxwell GPU) — inference latency and throughput measured on these devices.

Evaluation Metrics: We measured:

Table 2 updates the performance comparison with standard deviations and statistical significance (p-values from paired t-tests against TAHA). TAHA achieves comparable accuracy to full-head ViT while reducing computational costs by 38–42%, with statistically significant improvements over DANN (p<0.05) and RHP (p<0.01) on all datasets.

Figure 2 shows the evolution of HTS values during the alignment process. Three different patterns are observed: (1) Heads with uniformly high HTS (for instance, H1, H7) capture features common across species, such as leaf venation, (2) Heads starting with low scores but showing improvement (e.g., H4) adjust to traits specific to the target, and (3) Heads with consistently low scores (e.g., H11) encode patterns unique to the source and are later removed.

The correlation between alignment performance and phenotypic contribution is further illustrated in Figure 3. Retained heads are concentrated in the high-DAL/low-correlation quadrant, which supports the effectiveness of our joint scoring approach in Equation 6.

Figure 3 illustrates the correlation between DAL and phenotypic contribution. Retained heads (green dots) cluster in the low-DAL/high-correlation quadrant (DAL<0.8, Corr>0.4), indicating strong alignment and high phenotypic relevance. Pruned heads (red dots) are either high-DAL (poor alignment) or low-correlation (low phenotypic value).

Our pruning approach achieves a 40.9% average reduction in FLOPs while preserving 98.1% of the original model’s accuracy. On real edge devices: Raspberry Pi 4 achieves 14.1 ± 0.6 ms inference latency and 71 samples/second throughput; NVIDIA Jetson Nano achieves 8.3 ± 0.4 ms latency and 120 samples/second—meeting real-time requirements for field phenotyping (≤20 ms latency). Table 3 breaks down the computational savings across different model components.

We conducted ablation studies to validate key design choices:

Scoring Components: The exclusion of either the DAL or correlation term from HTS leads to accuracy declines of 4.7% and 3.2% respectively, which underscores the necessity of including both metrics.

Pruning Threshold: Adjusting β in Equation 7 indicates peak performance at β=1.5, while aggressive pruning at β=2.0 leads to reduced accuracy and conservative pruning at β=1.0 produces only modest efficiency improvements.

Iterative Alignment: The incremental method (progressively raising λ) achieves a 2.9% higher mAP than fixed-weight training, which indicates the advantage of stepwise phenotypic constraint application.

Table 4 summarizes hyperparameter sensitivity and transferability. α=0.6, β=1.5, and λ=0.1 (warmup) generalize across most species pairs (Arabidopsis→Maize, Tomato→Wheat) with ≤1.2% mAP variation. For distant transfers (Arabidopsis→Pine), adjusting α=0.7 (prioritize alignment) improves mAP by 2.1% (64.2→66.3), indicating dataset-specific tuning is only needed for phylogenetically distant species.

Although our approach shows robust results in transferring attention heads between plant species, several constraints merit examination. Initially, the existing framework presumes a common phenotypic trait space for both source and target species, an assumption that might be invalid for phylogenetically distant plants (e.g., Arabidopsis→pine). Supplementary experiments confirm this: the 7.2% accuracy drop in distant transfers is due to limited trait overlap (e.g., leaf vs. needle morphology), where no single attention head can capture both structures. The morphological gap between such species could exceed the adaptation capacity of any attention head, regardless of pruning strategy. Second, the dynamic pruning threshold, though effective, introduces an extra hyperparameter (β) necessitating meticulous adjustment to achieve the best performance for varying species pairs. Empirical findings indicate β=1.5 performs effectively for the majority of angiosperms, yet transfers between monocots and dicots may require distinct parameters. Third, the present approach assesses head transferability at the species level, which may fail to account for differences within species due to environmental influences such as drought or nutrient stress. These limitations suggest opportunities for refinement in future work.

The concepts forming the basis of our method, adaptable attention head detection and pruning tailored to tasks, could apply to multiple associated fields. In ecological monitoring, similar methods could adapt plant phenotyping models to different ecosystems while preserving key vegetation indices. In agricultural robotics, pruned attention mechanisms could support real-time crop analysis on embedded systems with constrained computational capabilities. Beyond plants, the framework might apply to cross-species animal behavior analysis, where certain attention patterns could correspond to universal movement or interaction traits. Nevertheless, these implementations would need adjustments to address the unique aspects of every field, for instance the time-related component in behavioral studies or the spatial arrangement in woodland environment observation. Whether the transferability metrics can be generalized across such varied contexts remains a question for further research.

As with any agricultural technology, the deployment of lightweight phenotyping models raises important ethical questions. The productivity improvements resulting from pruning attention heads could broaden the availability of sophisticated phenotyping technologies for small-scale agricultural producers, contingent upon the foundational data encompassing varied cultivation practices. Existing plant image datasets predominantly concentrate on staple crops such as wheat and maize, which may introduce bias in models against less-researched yet nutritionally valuable species. Moreover, the energy savings from model pruning must be weighed against the environmental costs of data collection, particularly in field settings where imaging may require frequent equipment transportation. Future work should address these issues through inclusive dataset curation and lifecycle analysis of the full phenotyping pipeline, not just the computational components. Establishing guidelines for equitable technology transfer will be crucial as these methods move from research to real-world implementation.