BEGIN:VCALENDAR VERSION:2.0 PRODID:-//wp-events-plugin.com//7.2.3.1//EN TZID:America/New_York X-WR-TIMEZONE:America/New_York BEGIN:VEVENT UID:627@fds.yale.edu DTSTART;TZID=America/New_York:20241014T160000 DTEND;TZID=America/New_York:20241014T170000 DTSTAMP:20250916T142140Z URL:https://fds.yale.edu/events/sds-seminar-yury-polyanskiy-mit/ SUMMARY:S&\;DS Seminar: Yury Polyanskiy (MIT)\, “Clustering and phase transitions in transformers” DESCRIPTION:Abstract: Most of the recent advances in AI are based around a neural architecture known as Transformer. We describe the process of evol ution of representations through the layers of Transformer as an interacti ng particle system.  This dynamical system numerically exhibits an unusua l two time-scales behavior:in the initial (short) phase particles coalesce to form meta-stable clusters\, which continue to slowly move\, occasional ly merging\, until eventually collapsing to a single point. We will presen t rigorous results confirming this behavior in various degrees of generali ty. Joint work with B. Geshkovski\, N. Karagodin\, C. Letrouit and P. Rigo llet.\n\n\n\nBio: Yury Polyanskiy is a Professor of Electrical Engineering and Computer Science and a member of IDSS and LIDS at MIT.  Yury receive d M.S. degree in applied mathematics and physics from the Moscow Institute of Physics and Technology\, Moscow\, Russia in 2005 and Ph.D. degree in e lectrical engineering from Princeton University\, Princeton\, NJ in 2010. His research interests are in information theory\, statistical learning an d error-correcting codes.\n\n\n\nDr. Polyanskiy was elected IEEE Fellow (2 024)\, and won the 2020 IEEE Information Theory Society James Massey Award \, 2013 NSF CAREER award and 2011 IEEE Information Theory Society Paper Aw ard.\n CATEGORIES:FDS Events,Statistics & Data Science Seminar LOCATION:Yale Institute for Foundations of Data Science\, Kline Tower 13th Floor\, Room 1327\, New Haven\, CT\, 06511\, United States X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=Kline Tower 13th Floor\, Ro om 1327\, New Haven\, CT\, 06511\, United States;X-APPLE-RADIUS=100;X-TITL E=Yale Institute for Foundations of Data Science:geo:0,0 END:VEVENT BEGIN:VTIMEZONE TZID:America/New_York X-LIC-LOCATION:America/New_York BEGIN:DAYLIGHT DTSTART:20240310T030000 TZOFFSETFROM:-0500 TZOFFSETTO:-0400 TZNAME:EDT END:DAYLIGHT END:VTIMEZONE END:VCALENDAR