Impact of network topology on the performance of Decentralized Federated Learning
CoRR(2024)
摘要
Fully decentralized learning is gaining momentum for training AI models at
the Internet's edge, addressing infrastructure challenges and privacy concerns.
In a decentralized machine learning system, data is distributed across multiple
nodes, with each node training a local model based on its respective dataset.
The local models are then shared and combined to form a global model capable of
making accurate predictions on new data. Our exploration focuses on how
different types of network structures influence the spreading of knowledge -
the process by which nodes incorporate insights gained from learning patterns
in data available on other nodes across the network. Specifically, this study
investigates the intricate interplay between network structure and learning
performance using three network topologies and six data distribution methods.
These methods consider different vertex properties, including degree
centrality, betweenness centrality, and clustering coefficient, along with
whether nodes exhibit high or low values of these metrics. Our findings
underscore the significance of global centrality metrics (degree, betweenness)
in correlating with learning performance, while local clustering proves less
predictive. We highlight the challenges in transferring knowledge from
peripheral to central nodes, attributed to a dilution effect during model
aggregation. Additionally, we observe that central nodes exert a pull effect,
facilitating the spread of knowledge. In examining degree distribution, hubs in
Barabasi-Albert networks positively impact learning for central nodes but
exacerbate dilution when knowledge originates from peripheral nodes. Finally,
we demonstrate the formidable challenge of knowledge circulation outside of
segregated communities.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要