Newsroom
Interactive Learning
-
S&DS Seminar: Zhuoran Yang (Yale), “Unveiling In-Context Learning: Provable Training Dynamics and Feature Learning in Transformers”
Abstract: In-context learning (ICL) is a cornerstone of large language model (LLM) functionality, yet its theoretical foundations remain elusive due to the complexity of transformer architectures. In particular, most existing work only theoretically explains how the attention mechanism facilitates ICL under certain data models. It remains unclear how the other building blocks of the transformer contribute…

