Mean-Field Analysis for Learning Subspace-Sparse Polynomials with Gaussian Input
CoRR(2024)
摘要
In this work, we study the mean-field flow for learning subspace-sparse
polynomials using stochastic gradient descent and two-layer neural networks,
where the input distribution is standard Gaussian and the output only depends
on the projection of the input onto a low-dimensional subspace. We propose a
basis-free generalization of the merged-staircase property in Abbe et al.
(2022) and establish a necessary condition for the SGD-learnability. In
addition, we prove that the condition is almost sufficient, in the sense that a
condition slightly stronger than the necessary condition can guarantee the
exponential decay of the loss functional to zero.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要