We consider a federated UAV object detection framework consisting of a central server (base station) and multiple UAV clients. Each UAV is equipped with a certain amount of onboard image data and computational resources. The server coordinates the UAVs to collaboratively build the federated object detection model. Without sharing raw data, the system performs global aggregation to generate a high quality global object detection model.

The system, depicted in Figure 1, comprises a central server (base station) and multiple clients (UAVs). In each communication round, individual UAV clients train a local object detection model using onboard visual data and transmit the updated model parameters to the central server. Upon receiving the local models from all clients, the server aggregates them to update a global model, which is then disseminated to all UAVs for subsequent training rounds. This decentralized learning approach eliminates the necessity to exchange raw data, thereby safeguarding privacy. Let S represent the central server, and let the N participating UAVs be denoted as distributed clients within the system. Each client i possesses a local dataset D_i, with data distributions among clients typically being non-IID. Local training is conducted exclusively based on the dataset of each client.

The objective of each client i is to minimize the following local loss function during on device training:

where wi∈ℝd denotes the local model parameters of client i, and D_i represents the local dataset. Here, x_j is the input image and y_j is the corresponding annotated label including bounding boxes and class information. The term R(w_i) is a regularization function, such as L2 regularization, and β is a nonnegative scalar that controls the regularization strength. The function ζ_YOLO denotes the YOLOv8 object detection loss, which typically consists of three components: classification loss, objectness confidence loss, and bounding box regression loss.

Specifically, the classification loss, objectness confidence loss, and bounding box regression loss are defined as follows:

where C is the number of object categories, yj(c) is the ground truth label for class c, and q^j(c) is the predicted class probability. yjobj∈{0,1} indicates the presence of an object, and q^jobj is the predicted objectness score. A_int denotes the intersection area between the predicted and ground truth bounding boxes, and A_uni denotes their union area.

After each communication round, the central server collects updated local model parameters from all clients and aggregates them to update the global model using a weighted averaging strategy: w=∑i=1Npiwi, where w denotes the global model parameters, and the weight p_i is determined by the relative size of each client's local dataset:pi=|Di|∑n=1N|Dn|, the overall optimization objective of the system is to minimize the global empirical risk:

where F_i(w) represents the local loss function of client i evaluated on the global model w. This objective ensures that the aggregated model is jointly optimized across all clients, with contributions weighted by data volume, thereby producing a globally shared model that strikes the best compromise in applicability across all clients.

We introduce a supervised self-distillation framework for individual client detectors, comprising five consecutive stages: an input block, a bounding box alignment block, a global class probability alignment block, a channel semantic alignment block, and an output block. Raw images remain within the UAV client and knowledge transfer occurs from both the historical personalized local model and the global model (teacher) to the local model (student) through the three alignment blocks delineated below.

(1) Input block After standard preprocessing, each minibatch is forwarded through both the historical local model (teacher) and the current local model (student) via parallel forward propagation. The historical local model is retained from previous communication rounds and serves as a personalized knowledge reservoir. This process yields corresponding classification logits {zT(j)},{zS(j)}∈ℝC for each detection token j = 1, …, M, which are used in the global class probability alignment loss ζ_G in Equation 8. Simultaneously, it extracts intermediate feature maps {FℓT},{FℓS} from head layer ℓ, where Fℓ∈ℝCℓ×Hℓ×Wℓ denotes the activation tensor at head layer ℓ; Together, the generated outputs including classification logits, objectness scores, and spatial features form the basis for subsequent knowledge alignment.

(2) Bounding Box Alignment block To distill localization knowledge, we align the regression predictions between the teacher and student models. Specifically, let the predicted normalized bounding box coordinates for each detection token j be denoted as bT(j)=[xT(j),yT(j),wT(j),hT(j)] and bS(j)=[xS(j),yS(j),wS(j),hS(j)] for the teacher and student models, respectively. The bounding box alignment loss is computed as:

where ||·||₂ denotes the ℓ₂ norm. The vector b^(j)∈ℝ⁴ contains the center coordinates (x, y), width w, and height h of the bounding box for token j.

This loss encourages the student model to produce bounding boxes that are spatially consistent with the teacher's regression outputs, thereby improving localization accuracy under non-IID data distributions.

(3) Global class-probability alignment block Let zT(j),zS(j)∈ℝC denote the teacher and student classification logits of the j-th detection token. Following temperature scaling, the alignment loss is as shown in Equation 8.

where σ(·) is the sigmoid (multi-label) or softmax (single-label) function, C is the number of classes, τ_G>0 is the temperature, and KL(·∥·) denotes the Kullback-Leibler divergence. Minimizing Equation 8 transfers high-level semantic priors from teacher to student.

(4) Channel-semantic alignment block To further improve spatial awareness, we align the relative channel importances between teacher and student over head layer ℓ (including three feature maps P3, P4, and P5). This process consists of two steps: extracting spatial channel means and then computing a KL divergence over their softmax scaled distributions.

To align the channel semantics, we apply temperature scaled softmax followed by KL divergence, as shown in Equation 9:

where ε_a∈ℝ is a layer specific importance coefficient used to emphasize different feature map layers (P3, P4, and P5) during the channel semantic alignment process. Here, a = 1, 2, 3 correspond to the three feature maps at different spatial resolutions, and ℓ_a denotes the associated head layer in the YOLOv8 architecture. The feature tensor at layer ℓ_a is denoted as Fℓa∈ℝCℓa×Hℓa×Wℓa. To improve alignment effectiveness, we assign larger weights to lower level features by setting ε₁>ε₂>ε₃, which strengthens the alignment focus on P3 and enhances the detection of small objects commonly present in UAV imagery. The parameter τ_C>0 represents the temperature scaling factor used in the channel semantic alignment loss.

(5) Output block and unified objective Based on the above calculations, the student is optimized with a composite loss, as shown in Equation 10:

where ζ_CE is the standard classification loss, ζ_box is the bounding box regression term, and λ₁, λ₂, λ₃≥0 weight the alignment losses. This joint objective enforces global consistency while retaining local personalization, thus alleviating overfitting and drift in heterogeneous federated environments.

In federated UAV object detection, the disparate distribution of images among devices results in incongruent local optimization goals, leading to divergent model update trajectories and subsequent model drift. To mitigate this issue, FL-JSDDC proposes the incorporation of a local drift variable for modeling and correction. Each client is tasked with managing a drift vector φ_i = w−w_i to monitor the variance between its local model and the prevailing global model, subsequently fine tuning the model parameters prior to aggregation. To provide an approximate assessment of the local model drift, a drift regularization term is introduced as follows:

Each client uses this regularization term and the empirical loss term on its corresponding dataset to train both the model parameters and the local drift variable, thus converting the constrained optimization problem into an unconstrained one.

In FL-JSDDC, the drift compensation loss for each client consists of two components: the drift regularization term and the gradient correction term. The drift regularization term is used to penalize the deviation of the local model from the global model, while the gradient correction term adjusts the local model toward the global model by correcting the local gradient during the training process, as shown in Equation 12:

where, α is the weight coefficient for the drift regularization term, and ξ_i controls the gradient correction term for stochastic gradient optimization. To smooth out gradient heterogeneity, we set the gradient correction term as ξi(φi;μi,μ)=1ηE〈φi,φi-φ〉, where η is the learning rate, and E is the number of training iterations in one round. μ_i is the local update of the i-th client's local parameters in the previous round, and μ is the average update of all clients' local parameters in the previous round. In the t-th round, we have: μit=wit-wit-1 and μt=1N∑i=1Nμit, where wit and wit-1 are the local model parameters of client i in the t-th and (t−1)-th rounds, respectively. The purpose of this term is to reduce the variance of the local gradients.

Next, we introduce the update rule for the local drift variable φ_i. Taking the t+1-th iteration as an example, in FL-JSDDC, the local drift variable tracks the gap between the local model and the global model. In the t+1-th round of training, we assume that the global model parameters are updated to w^t+1, while the local model parameters remain fixed. Then we can update the local drift variable using φit+1=φit+(wt+1-wt), but due to the unavailability of global data, it is impossible to directly update the global model.

Assume we first update the local model parameters from wit to wit+1. Then we consider the following two points: 1) At the beginning of each round, the local model parameters are assigned to the global model parameters: wit=wt. 2) For client i, the local model parameters wit+1 are an estimate of the updated global model parameters w^t+1. Thus, we can approximate the update of the local drift variable as:

In this way, we use the update of the local model parameters to adjust the local drift variable (Algorithm 1 and Figure 3).

To update the global model parameters, each client uses its local drift variable (φit+1+wit+1) to correct the local model parameters before model aggregation. Then, each client uploads the corrected local parameters to the server. Similar to FedAvg, the server performs a weighted average of the corrected local parameters to obtain the global model parameters:

where D is the global dataset. The final joint local objective function in FL-JSDDC is formulated as:

By jointly integrating self-distillation and drift compensation, FL-JSDDC enhances both knowledge transfer and parameter alignment during local training. This leads to faster convergence and improved detection performance under non-IID data distributions.

Table 1 compares the number of communication rounds (R#) required for FL-JSDDC and seven baseline methods (FedAvg, FedNova, FedProx, FedPer, FedMIX, FedYolo and ACSFed), along with two ablation variants, FL-SD and FL-DC to reach the target detection performance of 21%. All experiments are conducted on the VisDrone2019-DET dataset under three data distribution settings: IID, ND(1)(mild non-IID), and ND(2)(severe non-IID).

In the IID setting, data is uniformly distributed, and local datasets are similar across clients. As a result, the divergence between local and global models is minimal, yielding robust training. For instance, with 15 clients, FedAvg requires 13 rounds to reach the target detection performance due to its simple weighted averaging strategy, which fails to correct client model deviations. FedNova slightly improves convergence by normalizing updates, but still needs 15 rounds. FedProx introduces a proximal term to restrict update deviation, reducing the round count to 10, but inconsistencies in update directions remain. FedPer adjusts global updates for better robustness, yet converges in 12 rounds, on par with FedAvg. FedMIX leverages a mixing strategy to fuse local and global models, achieving modest improvement with 11 rounds. FedYolo and ACSFed achieve convergence in 12 and 11 rounds respectively under IID settings, but their advantages in generalization and adaptive selection are less pronounced due to the statistical similarity among clients.

The ablation study conducted by FL-SD focuses solely on self-distillation, demonstrating convergence within 10 rounds under IID settings, suggesting that incorporating soft guidance from the global model facilitates convergence. However, the absence of explicit constraints on model drift impedes FL-SD from achieving optimal speed. On the other hand, FL-DC, which exclusively implements drift compensation, achieves convergence in 12 rounds, effectively mitigating drift but lacking personalized feature learning. Integration of both mechanisms in FL-JSDDC yields superior detection performance, reaching the target in only 6 rounds. This outcome underscores the significance of combining self-distillation and drift compensation for rapid and robust convergence. A comparison between FL-SD and FL-JSDDC underscores the substantial enhancement in global alignment through drift correction, while the disparity between FL-DC and FL-JSDDC underscores the critical role of knowledge transfer in ensuring resilient representation learning.

To further validate the robustness of our method, we conduct ablation studies by separately evaluating: FL-DC, which applies only drift compensation; FL-SD, which applies only self-distillation; and the full FL-JSDDC framework that combines both modules.

In the context of ND(1) settings involving 15 participating clients exhibiting moderate heterogeneity, the convergence speed deteriorates across all baseline methods. Specifically, FedAvg, FedNova, and FedPer necessitate 16, 19, and 14 rounds, respectively, while FedProx and FedMIX demonstrate slightly improved performance, requiring 11 and 13 rounds, respectively. FedYolo and ACSFed converge in 15 and 13 rounds, respectively. The escalation in round requirements for convergence with even minor heterogeneity underscores the challenge. Notably, FL-SD and FL-DC exhibit comparable performance to the aforementioned baselines, suggesting limited efficacy of a singular approach. In contrast, the integration of FL-JSDDC, incorporating both modules, achieves notably enhanced convergence efficiency.

In the ND(2) scenario involving 15 client participants with diverse data distributions, convergence performance notably deteriorates across established methods. Specifically, both FedAvg and FedPer necessitate 23 and 30 communication rounds, respectively, to attain the desired detection performance level (21% mAP@50). FedYolo achieves convergence in 24 rounds, while ACSFed does so in 26 rounds. Conversely, FedNova, FedProx, and FedMIX fall short of meeting the performance target within the 50 round threshold, underscoring their vulnerability to data distribution imbalances. These findings underscore the substantial impediment posed by significant data heterogeneity on model aggregation and training resilience. Notably, FL-JSDDC accomplishes the performance goal in a mere 15 rounds, showcasing exceptional adaptability and robustness in non-IID settings.

The detection performance is influenced by the scale of clients. A smaller number of clients, such as 5, leads to less fragmented data, lower heterogeneity, and quicker convergence across all methods. Notably, FL-JSDDC demonstrates the fastest convergence under these circumstances. Conversely, with 35 clients, data fragmentation increases, leading to higher heterogeneity and decreased convergence efficiency. Most baseline methods experience notable slowdowns, particularly FedAvg and FedNova. Even in this challenging scenario, FL-JSDDC maintains its superior speed and robustness. This resilience is attributed to its design, wherein self-distillation allows clients to extract globally aligned features from limited local data, and drift compensation regulates local deviations from the global model. These mechanisms enable FL-JSDDC to uphold synchronization and robust convergence, even in demanding conditions characterized by a large number of clients and significant data heterogeneity.

Table 2 reports the best detection performance achieved by FL-JSDDC under various settings and compares it against multiple baseline methods. Experiments are conducted on the VisDrone2019-DET dataset across different client scales (5, 15, 35) and data distribution settings (IID, ND(1), ND(2)).

In the IID scenario with uniformly distributed client data, model convergence exhibits greater resilience and leads to enhanced overall detection performance. For example, when considering 5 clients, FL-JSDDC demonstrates an mAP@50 of 32.61%, showcasing a significant improvement over FedAvg (28.44%), and FedProx (29.94%). Additionally, FedYolo and ACSFed achieve 28.72% and 29.34%, respectively. As the client count increases, FL-JSDDC sustains strong detection performance, achieving 28.49% with 35 clients, surpassing FedAvg (24.32%) and FedNova (25.49%). These results underscore not only the algorithm's robustness under balanced data distributions but also its scalability.

In the ND(1) scenario involving 5 participating clients with moderate data heterogeneity, conventional methods exhibit significant performance deterioration. Specifically, FedAvg and FedNova experience a decrease in mAP@50 to 23.21% and 24.90%, respectively, while FedYolo and ACSFed achieve 23.84% and 23.37%, respectively, showing a further decline as the number of clients increases. In contrast, FL-JSDDC demonstrates consistent detection performance, attaining 29.15% with 5 clients and 27.92% with 15 clients. These findings suggest that FL-JSDDC effectively mitigates the effects of mild data heterogeneity and sustains robust detection performance across varying numbers of clients.

In the ND(2) scenario characterized by heightened heterogeneity, conventional methods exhibit notable declines in detection accuracy. Specifically, when dealing with 35 clients, FedAvg and FedNova achieve only 16.23% and 15.02% accuracy, while FedProx and FedPer drop to 14.94% and 15.76%, respectively. In contrast, FedYolo and ACSFed attain 16.44% and 16.41% accuracy, respectively, in the same setting. Despite these formidable challenges, FL-JSDDC stands out by achieving 20.27% accuracy in this extreme scenario, surpassing all baseline methods and showcasing remarkable resilience to highly non-IID conditions.

From the perspective of client scale, increasing the number of clients from 5 to 35 leads to more fragmented data and greater training difficulty. While most baseline methods suffer significant detection performance degradation, FL-JSDDC maintains a high level of detection performance. For instance, FL-JSDDC achieves 24.91% mAP@50 with 5 clients and still retains 20.27% with 35 clients, markedly outperforming all other methods. This result further validates its ability to cope with the increased heterogeneity introduced by larger client populations.

Overall, FL-JSDDC consistently delivers superior detection performance across all data partition settings and client scales. This improvement is primarily attributed to the synergistic effects of its self-distillation and drift compensation mechanisms. The self-distillation component guides local models to learn globally consistent features, improving their adaptability. Meanwhile, the drift compensation component effectively suppresses excessive local model drift, enhancing the robustness of global aggregation. Together, these mechanisms enable FL-JSDDC to maintain high detection performance and robust convergence even in challenging federated environments, making it a promising solution to the client drift problem in federated object detection.

To further demonstrate the convergence behavior and robustness of the proposed method, Figure 4 presents the detection performance curves of FL-JSDDC and baseline methods over 50 communication rounds under three data heterogeneity settings. All results are obtained under a federated configuration involving 5 participating clients. In the IID setting (Figure 4a), where client data distributions are balanced, FL-JSDDC rapidly achieves high detection performance, outperforming other methods from the early stages of training.

Under the ND(1) setting (Figure 4b), where client distributions slightly differ, FL-JSDDC still maintains a clear advantage in both convergence speed and robustness, while FedAvg and FedNova exhibit slower and more volatile trends.

In the most challenging ND(2) scenario (Figure 4c), where class imbalance across clients is severe, baseline methods suffer significant detection performance degradation and stagnation. However, FL-JSDDC demonstrates strong resilience, achieving higher detection performance with faster convergence, validating the robustness of its joint self-distillation and drift compensation design in addressing model drift and local overfitting in non-IID environments.