Federated Class-Incremental Learning with Prototype Guided Transformer
CoRR(2024)
摘要
Existing federated learning methods have effectively addressed decentralized
learning in scenarios involving data privacy and non-IID data. However, in
real-world situations, each client dynamically learns new classes, requiring
the global model to maintain discriminative capabilities for both new and old
classes. To effectively mitigate the effects of catastrophic forgetting and
data heterogeneity under low communication costs, we designed a simple and
effective method named PLoRA. On the one hand, we adopt prototype learning to
learn better feature representations and leverage the heuristic information
between prototypes and class features to design a prototype re-weight module to
solve the classifier bias caused by data heterogeneity without retraining the
classification layer. On the other hand, our approach utilizes a pre-trained
model as the backbone and utilizes LoRA to fine-tune with a tiny amount of
parameters when learning new classes. Moreover, PLoRA does not rely on
similarity-based module selection strategies, thereby further reducing
communication overhead. Experimental results on standard datasets indicate that
our method outperforms the state-of-the-art approaches significantly. More
importantly, our method exhibits strong robustness and superiority in various
scenarios and degrees of data heterogeneity. Our code will be publicly
available.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要