BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//wp-events-plugin.com//7.4.0.1//EN
TZID:America/New_York
X-WR-TIMEZONE:America/New_York
BEGIN:VEVENT
UID:824@fds.yale.edu
DTSTART;TZID=America/New_York:20241218T150000
DTEND;TZID=America/New_York:20241218T160000
DTSTAMP:20250916T142146Z
URL:https://fds.yale.edu/events/fds-seminar-blake-bordelon-harvard/
SUMMARY:FDS Seminar: Blake Bordelon (Harvard)
DESCRIPTION:\n"Scaling Limits and Scaling Laws of Deep Learning" \n\n\n\nA
 bstract: Scaling up the size and training horizon of deep learning model
 s has enabled breakthroughs in computer vision and natural language proces
 sing. Empirical evidence suggests that these neural network models are des
 cribed by regular scalinglaws where performance of finite parameter mode
 ls improves as model size increases\, eventually approaching a limit descr
 ibed by the performance of an infinite parameter model. In this talk\, we 
 will first examine certain infinite parameter limits of deep neural networ
 ks which preserve representation learning and then describe how quickly fi
 nite models converge to these limits. Using dynamical mean field theory me
 thods\, we provide an asymptotic description of the learning dynamics of r
 andomly initialized infinite width and depth networks. Next\, we will empi
 rically investigate how close the training dynamics of finite networks are
  to these idealized limits. Lastly\, we will provide a theoretical model o
 f neural scalinglaws which describes how generalization depends on three
  computational resources: training time\, model size and data quantity. Th
 is theory allows analysis of compute optimal scaling strategies and pred
 icts how model size and training time should be scaled together in terms
  of spectral properties of the limiting kernel. The theory also predicts h
 ow representation learning can improve neural scalinglaws in certain reg
 imes. For very hard tasks\, the theory predicts that representation learni
 ng can approximately double the training-time exponent compared to the sta
 tic kernel limit.\n\n\n\nBio: Blake Bordelon is a PhD student in Applied M
 ath at Harvard University where he researches the theory of natural and ar
 tificial neural networks. \n
CATEGORIES:FDS Events,Postdoctoral Applicants
LOCATION:t\, \, 
GEO:0.000000;0.000000
X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=\, ;X-APPLE-RADIUS=100;X-TI
 TLE=t:geo:0.000000,0.000000
END:VEVENT
BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:STANDARD
DTSTART:20241103T010000
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
TZNAME:EST
END:STANDARD
END:VTIMEZONE
END:VCALENDAR