Academy of Mathematics and Systems Science, CAS
Colloquia & Seminars

Speaker:	王立威教授，北京大学智能学院
Inviter:
Title:	Robust Generalization Requires Exponentially Large Models
Language:	Chinese
Time & Venue:	2023.05.12 15:00 N109
Abstract:	It is well-known that modern neural networks are vulnerable to adversarial examples. To mitigate this problem, a series of robust learning algorithms have been proposed. However, although the robust training error can be near zero via some methods, all existing algorithms lead to a high robust generalization error. In this talk, I will provide a theoretical understanding of this puzzling phenomenon from the perspective of expressive power for deep neural networks. Specifically, for binary classification problems with well-separated data, we show that, for ReLU networks, while mild over-parameterization is sufficient for high robust training accuracy, there exists a constant robust generalization gap unless the size of the neural network is exponential in the data dimension d. This result holds even if the data is linear separable.