BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Claremont Center for the Mathematical Sciences - ECPv6.18.1//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:Claremont Center for the Mathematical Sciences
X-ORIGINAL-URL:https://colleges.claremont.edu/ccms
X-WR-CALDESC:Events for Claremont Center for the Mathematical Sciences
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20250309T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20251102T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20260308T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20261101T090000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20270314T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20271107T090000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20261102T161500
DTEND;TZID=America/Los_Angeles:20261102T171500
DTSTAMP:20261007T205549Z
CREATED:20261007T204617Z
LAST-MODIFIED:20261007T205549Z
UID:10002365-1793636100-1793639700@colleges.claremont.edu
SUMMARY:In-Context Kernel Ridge Regression with Transformers (Mingsong Yan\, UCSB)
DESCRIPTION:Large language models (LLMs) can use examples provided in a prompt to make predictions for new queries without updating their parameters\, a capability known as in-context learning. In this talk\, I will explore how transformers\, the architectural backbone of many LLMs\, can perform nonlinear kernel ridge regression (KRR). I will first present numerical observations showing that transformers pretrained on regression tasks match KRR predictions more closely as depth increases. Motivated by these observations\, I will present a theoretical construction showing how a transformer can approximate the Gaussian KRR predictor through an intuitive connection to a convergent iterative solver. This construction has a natural interpretation: successive transformer layers approximate steps of the solver\, progressively refining the solution to the underlying kernel system. Together\, these results provide theoretical and empirical insights into in-context learning with transformers for nonlinear regression.
URL:https://colleges.claremont.edu/ccms/event/in-context-kernel-ridge-regression-with-transformers/
LOCATION:Estella 1021 (Emmy Noether Room)\, Pomona College\, Claremont\, CA\, 91711\, United States
CATEGORIES:Applied Math Seminar
ORGANIZER;CN="Ami Radunskaya":MAILTO:aradunskaya@pomona.edu
END:VEVENT
END:VCALENDAR