https://sparsenotes.com/tags/classic/ Recent content in classic on Sparse Notes https://sparsenotes.com/images/og-default.png https://sparsenotes.com/images/og-default.png Hugo -- gohugo.io Sat, 23 May 2026 00:00:00 +0000 https://sparsenotes.com/posts/2026/05/karpathy-lets-build-gpt/ Sat, 23 May 2026 00:00:00 +0000 https://sparsenotes.com/posts/2026/05/karpathy-lets-build-gpt/ Notes on Andrej Karpathy's 2023 classic 'Let's build GPT: from scratch, in code, spelled out.' — tokenization, the bigram baseline, the math of self-attention, multi-head, residual + layernorm, and how the same ~200 lines scale to a real GPT.