讲座题目：《Heavy-Tail Phenomena in Machine Learning》

主讲嘉宾：Lingjiong Zhu(朱凌炯)

讲座主持：荆中博 副教授

讲座时间：2025年6月24日下午两点

讲座地点：学院南路学术会堂604

讲座摘要：Tochastic gradient descents (SGD) are workhorse methods for solving large-scale optimization problems that arise in machine learning. Heavy-tail phenomena in SGD have been reported in several empirical studies. Experimental evidence in previous works suggests a strong interplay between the heaviness of the tails and generalization behavior of SGD. To address this empirical phenomenon theoretically, we establish novel links between the tail behavior and generalization properties of SGD through the lens of algorithmic stability. We show that the generalization error decreases as the tails become heavier, as long as the tails are lighter than a threshold. Moreover, we investigate the origins of the heavy tails in SGD. We show that the iterates can converge to a stationary distribution that is heavy-tailed with infinite variance. We further characterize the behavior of the tails with respect to algorithm parameters, the dimension, and the curvature. We support our theory with experiments conducted on synthetic data, fully connected, and convolutional neural networks.

嘉宾简介：Short Bio: Lingjiong Zhu received BA in Mathematics from University of Cambridge in 2008, and PhD in Mathematics under the supervision of Prof. S.R.S. Varadhan from Courant Institute of Mathematical Sciences at New York University in 2013. After working at Morgan Stanley and University of Minnesota, he joined Florida State University in 2015 as an Assistant Professor, where he is currently a Professor and holds the Thinking Machines Eminent Scholar Chair. His research interests include applied probability, data science, financial engineering and operations research. He has published in Annals of Applied Probability, Bernoulli, Finance and Stochastics, ICML, INFORMS Journal on Computing, Journal of Machine Learning Research, NeruIPS, Operations Research, Production and Operations Management, SIAM Journal on Financial Mathematics, Stochastic Processes and their Applications, Review of Economics and Statistics and many other outlets. He received Kurt O. Friedrichs Prize for an outstanding dissertation in mathematics from Courant Institute in 2013, Developing Scholar Award from Florida State University in 2022, Graduate Faculty Mentor Award from Florida State University in 2023, and MSOM iFORM SIG Best Paper Award from MSOM Society in 2023.

主办单位：管理科学与工程学院