Robust Decentralized Learning with Local Updates and Gradient Tracking
arxiv(2024)
摘要
As distributed learning applications such as Federated Learning, the Internet
of Things (IoT), and Edge Computing grow, it is critical to address the
shortcomings of such technologies from a theoretical perspective. As an
abstraction, we consider decentralized learning over a network of communicating
clients or nodes and tackle two major challenges: data heterogeneity and
adversarial robustness. We propose a decentralized minimax optimization method
that employs two important modules: local updates and gradient tracking.
Minimax optimization is the key tool to enable adversarial training for
ensuring robustness. Having local updates is essential in Federated Learning
(FL) applications to mitigate the communication bottleneck, and utilizing
gradient tracking is essential to proving convergence in the case of data
heterogeneity. We analyze the performance of the proposed algorithm,
Dec-FedTrack, in the case of nonconvex-strongly concave minimax optimization,
and prove that it converges a stationary point. We also conduct numerical
experiments to support our theoretical findings.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要