Ilya Sutskever – We're moving from the age of scaling to the age of research
Related videos
Grant Sanderson (@3blue1brown) – AI and the future of math
Dwarkesh Patel
75.9k views
The data black hole at the center of AI
Dwarkesh Patel
31.1k views
Chip design from the bottom up – Reiner Pope
Dwarkesh Patel
25.6k views
David Reich – Bronze Age shock, the Neanderthal puzzle, & farming’s sudden spread
Dwarkesh Patel
29.9k views
Dylan Patel — The Single Biggest Bottleneck to Scaling AI Compute
Dwarkesh Patel
58.6k views
The Department of War is making a huge mistake.
Dwarkesh Patel
35.5k views
Dario Amodei — “We are near the end of the exponential”
Dwarkesh Patel
41.0k views
What are we scaling?
Dwarkesh Patel
22.3k views
Sarah Paine – Why Russia Lost the Cold War
Dwarkesh Patel
136.4k views
“Kash Patel Is Suing Me For $5 Million Over A RETWEET!” – Elijah Schaffer
The Jimmy Dore Show
132.9k views
Ilya Sutskever on Model Generalization, Pre-training Limits, and SSI's Research Path
Discover why current AI models fail to generalize like humans and how SSI plans to pivot research beyond the scaling paradigm to achieve robust intelligence. This discussion details the critical bottlenecks in AI development and the necessary shift toward continual learning principles.
Short Summary
- Current AI success relies on scaling pre-training, but this method suffers from inadequate generalization compared to human capability.
- Reinforcement Learning (RL) training appears susceptible to reward hacking focused narrowly on evaluation benchmarks, leading to fragility.
- SSI views the field as returning to an “age of research,” requiring fundamental breakthroughs in learning mechanisms, not just increased compute.
- Ilya emphasizes that true AGI might be better defined as a continuously learning agent rather than a static, fully knowledgeable entity.
- SSI’s focus is an attempt to investigate superior learning principles, potentially leading to an AI robustly aligned to care for sentient life.
This conversation with Ilya Sutskever examines the limitations discovered following the scaling era of deep learning. We explore the disconnect between high evaluation scores and real-world performance, analyze the role of Value Functions, and contrast pre-training reliance against the sample efficiency of human learning. This breakdown focuses on identifying the next fundamental research challenge required for safe, scalable superintelligence.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Related videos
Grant Sanderson (@3blue1brown) – AI and the future of math
Dwarkesh Patel
75.9k views
The data black hole at the center of AI
Dwarkesh Patel
31.1k views
Chip design from the bottom up – Reiner Pope
Dwarkesh Patel
25.6k views
David Reich – Bronze Age shock, the Neanderthal puzzle, & farming’s sudden spread
Dwarkesh Patel
29.9k views
Dylan Patel — The Single Biggest Bottleneck to Scaling AI Compute
Dwarkesh Patel
58.6k views
The Department of War is making a huge mistake.
Dwarkesh Patel
35.5k views
Dario Amodei — “We are near the end of the exponential”
Dwarkesh Patel
41.0k views
What are we scaling?
Dwarkesh Patel
22.3k views
Sarah Paine – Why Russia Lost the Cold War
Dwarkesh Patel
136.4k views
“Kash Patel Is Suing Me For $5 Million Over A RETWEET!” – Elijah Schaffer
The Jimmy Dore Show
132.9k views
Top Comments (10)
god tier homepage refresh pull
“My cofounder said yes to Meta, and as a result he was able to enjoy a lot of near-term liquidity” has to be the most polite way of saying someone sold out
"Talent hits a target no one else can hit; Genius hits a target no one else can see."
Dwarkesh is the only big podcaster in this type of format to actually challenge the guests intellectually and not just nods everything off what the ai leaders say. that’s why he has the best guests come to him as well. great job dwarkesh :)
Ilya is so fundamentally likeable, honest and brilliant. This is in stark contrast to certain others in the AI game and listening to him feels peaceful. A deeply wise man.
(1) Happy to see Ilya doing well. (2) He's a breath of fresh air compared to other founders in the AI race. I learned a lot. (3) Congratulations, Dwarkesh, on both getting this rare, unicorn interview and for not shying away from asking some hard questions. Well done👏!
18:13 I can’t wrap my head around how the interviewer can hear Ilya cracking open such an interesting nut about how our emotions play into our reward function, and how our hunger “emotion” has ceased to serve us in a world with abundant food,and go “yup people have talking about scaling data” and launches into another topic. Ilya said a lot of really interesting things in this one, but that one really left me wanting to hear him expound upon how biological urges could map onto value functions.
Fun fact, this video is cited in Yann Lecun's recent paper on multimodal pretraining
Ilya is a true researcher in his heart and the one who truly cares about safety of human beings and human society. Respect for this man!!!
Love the sincerity from Ilya Sutskever..... and what stood out for me was that he said "We are back to research" Now I feel more motivated than ever to pursue my PHD in Computational Neuroscience.
Unlock the Data Inside
Turn Videos into Knowledge
- Get FREE 10/day: transcripts, summaries, chats
- Chat with videos, export text & PDF
- $1 free API credit for RAG, chatbots & research
Free forever plan • All features unlocked
Top Comments (10)
god tier homepage refresh pull
“My cofounder said yes to Meta, and as a result he was able to enjoy a lot of near-term liquidity” has to be the most polite way of saying someone sold out
"Talent hits a target no one else can hit; Genius hits a target no one else can see."
Dwarkesh is the only big podcaster in this type of format to actually challenge the guests intellectually and not just nods everything off what the ai leaders say. that’s why he has the best guests come to him as well. great job dwarkesh :)
Ilya is so fundamentally likeable, honest and brilliant. This is in stark contrast to certain others in the AI game and listening to him feels peaceful. A deeply wise man.
(1) Happy to see Ilya doing well. (2) He's a breath of fresh air compared to other founders in the AI race. I learned a lot. (3) Congratulations, Dwarkesh, on both getting this rare, unicorn interview and for not shying away from asking some hard questions. Well done👏!
18:13 I can’t wrap my head around how the interviewer can hear Ilya cracking open such an interesting nut about how our emotions play into our reward function, and how our hunger “emotion” has ceased to serve us in a world with abundant food,and go “yup people have talking about scaling data” and launches into another topic. Ilya said a lot of really interesting things in this one, but that one really left me wanting to hear him expound upon how biological urges could map onto value functions.
Fun fact, this video is cited in Yann Lecun's recent paper on multimodal pretraining
Ilya is a true researcher in his heart and the one who truly cares about safety of human beings and human society. Respect for this man!!!
Love the sincerity from Ilya Sutskever..... and what stood out for me was that he said "We are back to research" Now I feel more motivated than ever to pursue my PHD in Computational Neuroscience.