Super Weights in LLMs and the Failure of Selective Training
Weekly Paper Notes — one of the top picks from the 2026-07-11 CS paper digest. Area: AI / ML. Authors: Shreyas Subramanian, Adewale Akinfaderin, Akarsha Sehwag (Amazon) arXiv: 2607.08733 · PDF TL;DR “Super Weights” — individual scalar parameters in a large language model whose removal collapses task accuracy — were the interpretability finding of 2024–2025. The natural inference was that if these coordinates matter that much for the forward pass, they should also matter that much for learning: freeze everything else, train only the Super Weights (or a small neighbourhood around them), and you should get parameter-efficient fine-tuning for essentially free....