Date/Time
Thursday
16 Oct 2025
4:00 pm - 5:30 pm
Location
470 Stephens Hall
Event Type
Colloquium
Xuenan Cao
Assistant Professor, Department of Cultural and Religious Studies, The Chinese University of Hong Kong
This work argues that LLMs are machines of extrapolation. Extrapolation, a statistical function for predicting the next value in a series beyond the bound of the training data, contributes to both the success of generated pre-trained transformers (GPTs) in generating novel content and the controversies surrounding “hallucination” and the spread of misinformation. Machine learning research has evolved from the premise that neural networks do not often extrapolate. Yet competing theories demonstrate that high-dimensional models almost always extrapolate.
This paper argues that “extrapolation” is not only a point of technical contention, but has a historical origin in early cybernetics, which usefully illuminates certain aspects of LLM behavior today. In 1941, when Norbert Wiener transitioned from missile science to communication engineering, the pivotal concept he adopted was none other than “extrapolation.” Soviet mathematician Andrey Kolmogorov, known in part for a compression logic that inspired OpenAI, had developed a parallel extrapolation project in 1939. This article uncovers the connections between hot war science, Cold War cybernetics, and contemporary debates on LLM performances.
