Newsroom

Information Theory

  • FDS Colloquium: Jun’ichi Takeuchi (Kyushu), “Fisher information and Neural Tangent Kernels”

    Abstract: We argue relation between neural tangent kernels (NTK) and Fisher information matrices of neural networks. For the Fisher information matrices of two layer ReLU neural networks with random hidden weights, we demonstrated their approximate spectral decomposition, whose eigenvalue distribution highly concentrates (Takeishi et al. 2023). In particular, the sum of the top 3 eigenvalues…

  • S&DS Seminar: Aaditya Ramdas (CMU), “Bringing closure to FDR control: a general principle for multiple testing”

    Abstract: Since the publication of the seminal Benjamini-Hochberg paper (the most cited paper in statistics), it has been an open problem how the “closure principle” applies to controlling the false discovery rate (FDR). As background, the closure principle, formulated in a seminal 1976 paper, states that every procedure for controlling the familywise error rate (FWER) can…

  • S&DS Seminar: Jingfeng Wu (Berkeley), “Gradient Descent Dominates Ridge: A Statistical View on Implicit Regularization”

    Talk summary: A key puzzle in deep learning is how simple gradient methods find generalizable solutions without explicit regularization. This talk discusses the implicit regularization of gradient descent (GD) through the lens of statistical dominance. Using least squares as a clean proxy, we present two surprising findings.  First, GD dominates ridge regression. For any well-specified…

  • S&DS Seminar: Adam Smith (BU), “Privacy in Machine Learning and Statistical Inference”

    Zoom Link: https://yale.zoom.us/j/94223816617 Meeting ID: 942 2381 6617 Abstract: The results of learning and statistical inference reveal information about the data they use. This talk discusses the possibilities and limitations of fitting machine learning and statistical models while protecting the privacy of individual records. I will begin by explaining what makes this problem difficult, using…

  • FDS Colloquium: Song Mei (Berkeley), “Revisiting neural network approximation theory in the age of generative AI”

    Optional Zoom link: https://yale.zoom.us/j/97222935172 Abstract: Textbooks on deep learning theory primarily perceive neural networks as universal function approximators. While this classical viewpoint is fundamental, it inadequately explains the impressive capabilities of modern generative AI models such as language models and diffusion models. This talk puts forth a refined perspective: neural networks often serve as algorithm approximators,…

  • FDS Colloquium: Bento Natura (Columbia), “Faster Exact Linear Programming”

    Optional Zoom link: https://yale.zoom.us/j/99342713421 Abstract: We present a novel algorithm to solve various subclasses of linear programs, with a particular focus on strongly polynomial algorithms—those that operate in polynomial time relative to the problem’s dimension. Although subclasses like bipartite matching and maximum flow are known to be solvable in strongly polynomial time, the existence of…

View more