LLM Attention That Expands At Inference? Test Time Training Explained

Take your personal data back with Incogni! Use code bycloud at the link below and get 60% off an annual plan: https://incogni.com/bycloud RNN's hidden states be like: "You know, I am something of an ML model myself" check out my newsletter: https://mail.bycloud.ai/ Learning to (Learn at Test Time): RNNs with Expressive Hidden States [Paper] https://arxiv.org/abs/2407.04620 [Code PyTorch] https://github.com/test-time-training/ttt-lm-pytorch [Code JAX] https://github.com/test-time-training/ttt-lm-jax This video is supported by the kind Patrons & YouTube Members: 🙏Andrew Lescelius, alex j, Chris LeDoux, Alex Maurice, Miguilim, Deagan, FiFaŁ, Robert Zawiasa, Daddy Wen, Tony Jimenez, Panther Modern, Jake Disco, Demilson Quintao, Shuhong Chen, Hongbo Men, happi nyuu nyaa, Carol Lo, Mose Sakashita, Miguel, Bandera, Gennaro Schiano, gunwoo, Ravid Freedman, Mert Seftali, Mrityunjay, Richárd Nagyfi, Timo Steiner, Henrik G Sundt, projectAnthony, Brigham Hall, Kyle Hudson, Kalila, Jef Come, Jvari Williams, Tien Tien, BIll Mangrum, owned, Janne Kytölä, SO, Richárd Nagyfi, Hector, Drexon, Claxvii 177th, Inferencer, Michael Brenner, Akkusativ, Oleg Wock, FantomBloth [Discord] https://discord.gg/NhJZGtH [Twitter] https://twitter.com/bycloudai [Patreon] https://www.patreon.com/bycloud [Music] Massobeats - Noon [Profile & Banner Art] https://twitter.com/pygm7 [Video Editor] Silas