BEGIN:VCALENDAR VERSION:2.0 PRODID:-//wp-events-plugin.com//7.2.3.1//EN TZID:America/New_York X-WR-TIMEZONE:America/New_York BEGIN:VEVENT UID:880@fds.yale.edu DTSTART;TZID=America/New_York:20250922T153000 DTEND;TZID=America/New_York:20250922T170000 DTSTAMP:20250916T162815Z URL:https://fds.yale.edu/events/sds-seminar-jingfeng-wu-berkeley/ SUMMARY:S&\;DS Seminar: Jingfeng Wu (Berkeley)\, "Gradient Descent Domin ates Ridge: A Statistical View on Implicit Regularization" DESCRIPTION:\nTalk summary: A key puzzle in deep learning is how simple gra dient methods find generalizable solutions without explicit regularization . This talk discusses the implicit regularization of gradient descent (GD) through the lens of statistical dominance. Using least squares as a clean proxy\, we present two surprising findings.  First\, GD dominates ridge regression. For any well-specified Gaussian least squares problem\, the fi nite-sample excess risk of optimally stopped GD is no more than a constant times that of optimally tuned ridge regression. However\, there is a natu ral subset of these problems where GD achieves a polynomially smaller exce ss risk. Thus\, implicit regularization is statistically superior to expli cit regularization\, in addition to its computational advantages.\n\n\n\nS econd\, GD and online stochastic gradient descent (SGD) are incomparable. We construct a sequence of well-specified Gaussian least squares problems where optimally stopped GD is polynomially worse than online SGD\, and sim ilarly vice versa. Our construction leverages a key insight from benign ov erfitting\, revealing a fundamental separation between batch and online le arning.\n\n\n\nThis is joint work with Peter Bartlett\, Sham Kakade\, Jaso n Lee\, and Bin Yu.\n\n\n\nSpeaker bio: Jingfeng Wu is a Postdoctoral Fell ow at the Simons Institute for the Theory of Computing at UC Berkeley\, wh ere he is hosted by Peter Bartlett and Bin Yu. He is a member of the NSF/S imons Collaboration on the Theoretical Foundations of Deep Learning. Wu re ceived his Ph.D. in Computer Science from Johns Hopkins University\, where he was advised by Vladimir Braverman.\n\n\n\nWebsite. \n CATEGORIES:FDS Events,Statistics & Data Science Seminar LOCATION:Yale Institute for Foundations of Data Science\, Kline Tower 13th Floor\, Room 1327\, New Haven\, CT\, 06511\, United States X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=Kline Tower 13th Floor\, Ro om 1327\, New Haven\, CT\, 06511\, United States;X-APPLE-RADIUS=100;X-TITL E=Yale Institute for Foundations of Data Science:geo:0,0 END:VEVENT BEGIN:VTIMEZONE TZID:America/New_York X-LIC-LOCATION:America/New_York BEGIN:DAYLIGHT DTSTART:20250309T030000 TZOFFSETFROM:-0500 TZOFFSETTO:-0400 TZNAME:EDT END:DAYLIGHT END:VTIMEZONE END:VCALENDAR