Uniformly Stable Algorithms for Adversarial Training and Beyond
arxiv(2024)
摘要
In adversarial machine learning, neural networks suffer from a significant
issue known as robust overfitting, where the robust test accuracy decreases
over epochs (Rice et al., 2020). Recent research conducted by Xing et al.,2021;
Xiao et al., 2022 has focused on studying the uniform stability of adversarial
training. Their investigations revealed that SGD-based adversarial training
fails to exhibit uniform stability, and the derived stability bounds align with
the observed phenomenon of robust overfitting in experiments. This motivates us
to develop uniformly stable algorithms specifically tailored for adversarial
training. To this aim, we introduce Moreau envelope-𝒜, a variant of
the Moreau Envelope-type algorithm. We employ a Moreau envelope function to
reframe the original problem as a min-min problem, separating the non-strong
convexity and non-smoothness of the adversarial loss. Then, this approach
alternates between solving the inner and outer minimization problems to achieve
uniform stability without incurring additional computational overhead. In
practical scenarios, we show the efficacy of ME-𝒜 in mitigating the
issue of robust overfitting. Beyond its application in adversarial training,
this represents a fundamental result in uniform stability analysis, as
ME-𝒜 is the first algorithm to exhibit uniform stability for
weakly-convex, non-smooth problems.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要