Events
FDS Colloquium
Topics in Sequential Anytime-Valid Inference: Comparing Forecasters & Combining Evidence
 |
Speaker: Yo Joong "YJ" Choe Postdoctoral Scholar University of Chicago Wednesday, January 29, 2025 11:30AM - 1:00PM Lunch at 11:30am in room 1307
Talk from 12:00-1:00pm in room 1327A Location: Yale Institute for Foundations of Data Science, Kline Tower 13th Floor, Room 1327, New Haven, CT 06511 and via Webcast: https://yale.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=8ec2f9a0-1062-4bf1-9c92-b259013b41fb |
Abstract: Given sequentially observed data, anytime-valid methods guarantee valid inference at arbitrary stopping times, as opposed to pre-specified sample sizes, thereby allowing the experimenter to stop experiments early. In this talk, I will present two recent advances in the emerging field of sequential anytime-valid inference (SAVI).
First, consider two forecasters, each making a prediction for a sequence of events over time. How can we rigorously compare these forecasters as the events unfold, while avoiding restrictive assumptions such as stationarity? I will address this question by designing a novel inference procedure for estimating the time-varying difference in mean forecast scores. The procedure utilizes confidence sequences, which are sequences of confidence intervals that are anytime-valid and can be continuously monitored over time. I will demonstrate applications of this approach to real-world sports and weather forecasters.
Next, given a composite null hypothesis over sequentially observed data (e.g., whether high-volatility days are random in a financial time series), consider two or more testing procedures that are powerful against different alternatives. How can we combine these procedures so that we can leverage their collective statistical power, particularly when the procedures are valid under different information sets (i.e., filtrations)? This general question arises in various sequential inference problems, such as randomness testing and multi-step forecast evaluation. I will introduce a simple solution that allows us to combine arbitrary sequential tests that are based on e-processes—the SAVI notion of statistical evidence—across different filtrations.
Speaker bio: Dr. Yo Joong “YJ” Choe is a Postdoctoral Scholar at the University of Chicago’s Data Science Institute. He received a joint Ph.D. in Statistics and Machine Learning (ML) from Carnegie Mellon University in 2023. His research interests include: (1) reliable evaluation of black-box forecasters, with a focus on sequential anytime-valid inference (SAVI) methods, and (2) causal approaches to enhancing the transparency of large language models (LLMs). His work is published in top journals, such as Operations Research, as well as major ML conferences, such as NeurIPS, ICML, and ICLR. Previously, Dr. Choe was a Research Scientist at Kakao and Kakao Brain, where he developed models and resources for practical natural language processing (NLP) tasks.
Add To: Google Calendar | Outlook | iCal File
Submit an Event
Interested in creating your own event, or have an event to share? Please fill the form if you’d like to send us an event you’d like to have added to the calendar.