DWFed: A statistical- heterogeneity-based dynamic weighted model aggregation algorithm for federated learning - Chen, Aiguo; Fu, Yang; Wang, Lingfu; Duan, Guiduo (Frontiers in Neurorobotics, 2022)
Federated Learning is a distributed machine learning framework that aims to train a global shared model while keeping their data locally, and previous researches have empirically proven the ideal performance of federated learning methods. However, recent researches found the challenge of statistical heterogeneity caused by the non-independent and identically distributed (non-IID) data across devices, which leads to a significant decline in the performance of federated learning methods because the model divergence increases by training federated model with non-IID data. This statistical heterogeneity is common in distributed scenarios and thus dramatically restricts the application of federated learning methods. Therefore, statistical heterogeneity has become one of the critical challenges that current researchers of federated learning are confronted with. In this paper, a dynamic weighted model aggregation algorithm based on statistical heterogeneity for federated learning called DWFed is proposed, in which the index of statistical heterogeneity is firstly quantitatively defined through mathematical derivation. Then the index is used to calculate the weights of each local model for aggregating federated model, which is to constrain the model divergence caused by non-IID data. Multiple experiments on public benchmark data set reveal the improvements in performance and robustness of the federated models in heterogeneous settings.