Events
Postdoctoral Applicants
Statistics and strategic behavior in AI/ML evaluation
 |
Speaker: Chris Hays (MIT) PhD Candidate MIT Institute for Data, Systems and Society Monday, February 9, 2026 10:30AM - 11:30AM and via Webcast: https://yale.zoom.us/j/92138817301?pwd=DxCq6QCblWeqFaco10NY9ehBaPPPxk.1 |
Abstract: Machine learning systems have dramatically reorganized society, from what content we consume to whom we hire and how we work. To measure their impacts, we must build trustworthy evaluations. However, these systems are embedded in complex social environments, which means that evaluation data is rarely iid or exogenously determined. Thus, new statistical approaches are needed. In this talk, I’ll discuss two case studies in methods for ML evaluation. In the first, we analyze generative AI tournaments, which rank models based on pairwise human preferences. We’ll focus on standard ranking mechanisms’ non-robustness to strategic candidacy, where model providers choose which models to submit strategically in order to maximize the overall performance of their submissions, and propose a modification which does not suffer from this problem. In the second, we analyze a proposal from the discrimination law literature, which would require that ML model developers search for less discriminatory hiring, credit and housing models using randomness in training pipelines. We formulate this as a sequential decision-making problem and apply tools from anytime-valid inference to establish when a sufficient search has been conducted.
Bio: Chris Hays is a PhD candidate at the MIT Institute for Data, Systems and Society. He is interested in the theory of AI evaluations through the lens of statistics and strategic behavior. He is advised by Manish Raghavan. His PhD is supported by an NDSEG fellowship, and he has won several awards including a Best Paper award at WWW.
Zoom password: 123
Add To: Google Calendar | Outlook | iCal File
- Postdoctoral Applicants
Submit an Event
Interested in creating your own event, or have an event to share? Please fill the form if you’d like to send us an event you’d like to have added to the calendar.