learning-theory

Ilya Sutskever opens this Simons Institute lecture by admitting he almost gave the standard talk and decided against it. Instead he presents a set of results from years earlier at OpenAI, never published, on a question that had bothered him: supervised learning comes with a mathematical guarantee, and unsupervised learning does not. His claim is that it can, if you frame unsupervised learning as compression. This is the “Classic of the Week” pick, and it earns the label for a specific reason....

Weekly Paper Notes — one of the top picks from the May 24–30, 2026 CS paper digest. Area: NLP / Theory. Authors: Jon Kleinberg, Anay Mehrotra, Amin Saberi (Cornell / Yale / Stanford) arXiv: 2605.30324 · PDF TL;DR A line of theoretical work asks: given examples from an unknown target language drawn from a known countable collection, can a learner eventually output only new valid strings from that language? Prior results — including Kleinberg & Mullainathan’s 2024 paper that triggered the modern wave — assume the learner remembers the entire example history....