Who Should Own the Expert Cache? Kernel-Managed Tiering for Trillion-Parameter MoE Inference
Weekly Paper Notes — one of the top picks from the 2026-08-15 CS paper digest. Area: Operating Systems / Systems. Authors: Yuan Si (University of Waterloo), Yufeng Lin (Independent), Daming Li (Independent), Jialu Zhang (University of Waterloo, corresponding) arXiv: 2608.12103 · PDF TL;DR A trillion-parameter mixture-of-experts model routes each token through a small, input-dependent slice of its weights — in the production model studied here, an accepted token costs on average 1585 expert reads of 17....