BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//wp-events-plugin.com//7.2.3.1//EN
TZID:America/New_York
X-WR-TIMEZONE:America/New_York
BEGIN:VEVENT
UID:893@fds.yale.edu
DTSTART;TZID=America/New_York:20251015T113000
DTEND;TZID=America/New_York:20251015T130000
DTSTAMP:20251010T150605Z
URL:https://fds.yale.edu/events/fds-colloquium-junichi-takeuchi-kyushu/
SUMMARY:FDS Colloquium: Jun'ichi Takeuchi (Kyushu)\, "Fisher information an
 d Neural Tangent Kernels"
DESCRIPTION:\nAbstract: We argue relation between neural tangent kernels (N
 TK) and Fisher information matrices of neural networks. For the Fisher inf
 ormation matrices of two layer ReLU neural networks with random hidden wei
 ghts\, we demonstrated their approximate spectral decomposition\, whose ei
 genvalue distribution highly concentrates (Takeishi et al. 2023). In parti
 cular\, the sum of the top 3 eigenvalues (they have multiplicity) accounts
  for 97.7% of the sum of all the eigenvalues no matter how large the numbe
 r of parameters is. This means that the effective number of parameters is 
 almost dominated by the dimension of eigenspaces of the top 3 eigenvalues.
  In fact\, we derived a tight risk bound (Takeishi and Takeuchi 2024) us
 ing Barron and Cover theory for two-stage codes in the minimum description
  length (MDL) principle. Concerning our decomposition of Fisher informatio
 n matrices\, we recently found that the functions specified by the eigenve
 ctors with top 3 eigenvalues converge to spherical harmonic functions of o
 rders up to 2 as the number of parameters goes to infinity (Ho et al. 2025
 ). This result reveals a certain detailed aspect of the relation between N
 TK and Fisher information matrices\, since the NTK has a decomposition bas
 ed on spherical harmonic functions shown by Biettio and Mairal (2019). Bas
 ed on the decomposition\, they gave a nice kernel regression scenario in o
 ver-parameterization situation. We discuss the relation of Fisher informat
 ion and NTK of the two layer networks based on these spectral decompositio
 ns. Finally\, we show a vision to extend our result to more general feedfo
 rward neural networks.\n\n\n\nSpeaker bio: Jun'ichi Takeuchi was born in T
 okyo\, Japan in 1964. He graduated from the University of Tokyo in majorin
 g physics in 1989. He received the Dr. Eng. degree in mathematical enginee
 ring from the University of Tokyo in 1996. From 1989 to 2006\, he worked f
 or NEC Corporation\, Japan. In 2006\, he moved to Kyushu University\, Fuku
 oka\, Japan\, where he is a Professor. From 1996 to 1997 he was a Visiting
  Research Scholar at Department of Statistics\, Yale University\, New Have
 n\, CT\, USA. He was involved in the start of the annual workshop series o
 n Information-Based Induction Sciences (IBIS\, Japan domestic) in 1998. Hi
 s research interest includes information theory and machine learning. He i
 s a member of IEEE\, IEICE\, and JSIAM.\n
CATEGORIES:FDS Events,Colloquium
LOCATION:Yale Institute for Foundations of Data Science\, Kline Tower 13th 
 Floor\, Room 1327\, New Haven\, CT\, 06511\, United States
X-APPLE-STRUCTURED-LOCATION;VALUE=URI;X-ADDRESS=Kline Tower 13th Floor\, Ro
 om 1327\, New Haven\, CT\, 06511\, United States;X-APPLE-RADIUS=100;X-TITL
 E=Yale Institute for Foundations of Data Science:geo:0,0
END:VEVENT
BEGIN:VTIMEZONE
TZID:America/New_York
X-LIC-LOCATION:America/New_York
BEGIN:DAYLIGHT
DTSTART:20250309T030000
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
TZNAME:EDT
END:DAYLIGHT
END:VTIMEZONE
END:VCALENDAR