ai-ml

Weekly Paper Notes — one of the top picks from the 2026-08-29 CS paper digest. Area: AI / Machine Learning. Authors: Lukas Kuhn, Lucas Maes, Giuseppe Serra, Quentin Le Lidec, Yann LeCun, Randall Balestriero, Florian Buettner (German Cancer Research Center / DKTK / Goethe University Frankfurt, Mila, Université de Montréal, Brown University, Courant Institute NYU, AMI Labs) arXiv: 2608.27395 · PDF · Project page TL;DR Self-supervised video encoders have been expensive twice over: video carries an order of magnitude more tokens than an image, and the dominant methods add machinery on top of that cost purely to keep representations from collapsing....

Weekly Paper Notes — one of the top picks from the 2026-08-22 CS paper digest. Area: AI / ML. Authors: Nayeon Kim, Hojin Lee, Yunju Bak, Jaesun Park, Boseop Kim, et al. arXiv: 2608.20061 · PDF · Published at COLM 2026 TL;DR Choosing the learning rate for a frontier pretraining run is one of the highest-stakes, least-principled decisions in the field. At trillion-token scale a single sweep is unaffordable, so labs guess, extrapolate by folklore, or burn compute they’d rather spend on tokens....

Weekly Paper Notes — one of the top picks from the 2026-08-15 CS paper digest. Area: AI / ML. Authors: Julian Minder, Viktor Moskvoretskii, Raghav Singhal (equal contribution), Difan Jiao, Andy Arditi, Shaobo Cui, Jannik Brinkmann, Ashton Anderson, Roland Aydin, Robert West, and others — EPFL, MATS, University of Toronto, Saarland University, Northeastern, SJTU, DFKI, Ontocord AI, Hereon/TUHH arXiv: 2608.13482 · PDF · Models & data TL;DR Every production language model today learns what the world is like during pretraining and only learns who it is supposed to be afterwards, during post-training....

Weekly Paper Notes — one of the top picks from the 2026-07-25 CS paper digest. Area: AI / ML (Agent Training). Authors: Xiao Yu, Baolin Peng, Ruize Xu et al. arXiv: 2607.21557 · PDF TL;DR Modern agents are shaped less by their base model and more by the harness wrapped around it — Claude Code, Codex, OpenClaw, GUI-use scaffolds. But those harnesses are stateful, multi-process, and full of tool orchestration, which means existing open SFT/RL stacks (veRL, TRL, OpenRLHF) can’t natively express a rollout inside one....

Weekly Paper Notes — one of the top picks from the 2026-07-25 CS paper digest. Area: NLP / LLM Inference. Author: Alagappan Valliappan arXiv: 2607.21535 · PDF TL;DR Frontier LLMs increasingly ship a built-in Multi-Token-Prediction (MTP / NEXTN) draft head for speculative decoding, based on the assumption that the draft is negligibly cheap. Windowed-MTP shows that assumption breaks catastrophically at million-token context: the native MTP head does full attention over the entire KV cache at every draft step, so its cost grows linearly with context and comes to dominate — precisely where speculation is supposed to matter most....

Weekly Paper Notes — one of the top picks from the 2026-07-11 CS paper digest. Area: AI / ML. Authors: Shreyas Subramanian, Adewale Akinfaderin, Akarsha Sehwag (Amazon) arXiv: 2607.08733 · PDF TL;DR “Super Weights” — individual scalar parameters in a large language model whose removal collapses task accuracy — were the interpretability finding of 2024–2025. The natural inference was that if these coordinates matter that much for the forward pass, they should also matter that much for learning: freeze everything else, train only the Super Weights (or a small neighbourhood around them), and you should get parameter-efficient fine-tuning for essentially free....

Weekly Paper Notes — one of the top picks from the 2026-06-27 CS paper digest. Area: AI / ML. Authors: Nicklas Hansen, Xiaolong Wang (UC San Diego) arXiv: 2606.27326 · PDF · Interactive TL;DR Generative action-conditional world models produce visually fluent rollouts that drift from the true dynamics — the “fluent but wrong” failure mode familiar from LLMs, now in video. Hansen and Wang argue this is fundamentally a data coverage problem: hallucinations concentrate in regions of state-action space the training set under-samples, and cheap data-centric signals computed at inference time can both detect and prevent them....

Weekly Paper Notes — one of the top picks from the 2026-06-27 CS paper digest. Area: AI / ML. Authors: Josef Chen arXiv: 2606.27288 · PDF TL;DR The paper formalizes a hard accuracy ceiling for any multi-model LLM system whose final output is one member’s answer — routing, voting, cascades, fusion, mixture-of-agents. That ceiling is 1 − β, where β is the co-failure rate: the fraction of queries on which every model in the pool is wrong simultaneously....

Weekly Paper Notes — one of the top picks from the 2026-06-06 CS paper digest. Area: AI / ML. Authors: Akarsh Kumar, Phillip Isola (MIT) arXiv: 2606.06479 · PDF TL;DR This paper proposes Supervised Memory Training (SMT), a way to pretrain nonlinear RNNs without ever doing backpropagation through time (BPTT). The trick: replace recurrent credit assignment with a supervised problem over memory transitions. A Transformer-based “memory encoder” is first trained with a predictive-state objective — it learns a representation m_t that retains exactly the information about the past needed to predict the future....

Weekly Paper Notes — one of the top picks from the 2026-06-06 CS paper digest. Area: NLP / Systems-for-ML. Authors: Yutao Sun, Yanqi Zhang, Li Dong, et al. (Microsoft Research Asia) arXiv: 2606.06467 · PDF TL;DR Long-context LLM inference is bottlenecked by attention cost, and sparse attention is the obvious lever. The two existing families both disappoint in practice: block-sparse patterns (sliding window, dilated, etc.) give clean speedups but lose quality, while token-sparse patterns (top-k over the KV cache) preserve quality but spend most of the budget deciding which tokens to attend to — the routing itself becomes the bottleneck....