The ANTC of ChatGPT: On the Mathematical Foundations of Large Language Models (Gizem Karaali, Pomona)

Large Language Models like ChatGPT rely on surprisingly familiar mathematics. This talk will explore how ideas from (linear) algebra, number theory and combinatorics  appear — both directly and indirectly — in the structure and behavior of these models. Along the way, we’ll touch on themes like structure, symmetry, and scale, and consider how abstract mathematical ideas can shed light on systems that process and generate human language. The talk will be self-contained, and no background in machine learning will be assumed. (This abstract was prepared with the assistance of ChatGPT, which seems to be remarkably self-aware of its own mathematical foundations.)