Why Transformers Hit a Wall the Moment Physics Shows Up
Anima Anandkumar spent 83 minutes on Latent Space explaining a specific architectural claim: transformers are the wrong tool for physical simulation, and the reason is not compute budget but a mismatch in what the architecture assumes about its inputs. She has the receipts — her group’s weather model, FourCastNet, is deployed at a national weather agency and runs tens of thousands of times faster than the numerical models it replaces....