Some thoughts on the Sutton interview
Related videos
Chip design from the bottom up – Reiner Pope
Dwarkesh Patel
25.6k views
Gout Gout: The 60 Minutes Interview
60 Minutes
74.5k views
Kash Patel SNAPS at Reporter on Live TV
The Bulwark
126.2k views
Trump’s FOX Interview on Iran Was a Total Mess
The Bulwark
484.4k views
The Department of War is making a huge mistake.
Dwarkesh Patel
35.5k views
Dario Amodei — “We are near the end of the exponential”
Dwarkesh Patel
41.0k views
The Worst Interview of the Trump Administration So Far
The Bulwark
61.2k views
What are we scaling?
Dwarkesh Patel
22.3k views
Sarah Paine – Why Russia Lost the Cold War
Dwarkesh Patel
136.4k views
Ilya Sutskever – We're moving from the age of scaling to the age of research
Dwarkesh Patel
55.6k views
Analyzing Richard Sutton's Perspective on LLMs, Imitation Learning, and AGI Pathways
Understand the fundamental limitations Richard Sutton identifies in current LLM training paradigms, and assess how the speaker argues imitation learning might serve as a crucial, complementary step toward achieving AGI. This synthesis clarifies the divide between massive model scale and the necessity of continual, self-directed learning systems.
Short Summary
- [00:00:22] Sutton’s “Bitter Lesson” mandates focusing compute on scalable techniques rather than fixed human knowledge structures.
- [00:01:26] LLMs currently fail because they learn human responses from inelastic data, not self-directed engagement with truth.
- [00:02:49] The speaker refutes a strict dichotomy, positioning imitation learning as continuous with and complementary to Reinforcement Learning (RL).
This discussion breaks down Sutton’s critique concerning LLMs’ inability to learn robustly outside specialized training phases. It then presents a counter-argument suggesting that current pretraining models establish necessary, high-leverage foundational knowledge required for subsequent RL advancements.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Unlock all features
FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.
Related videos
Chip design from the bottom up – Reiner Pope
Dwarkesh Patel
25.6k views
Gout Gout: The 60 Minutes Interview
60 Minutes
74.5k views
Kash Patel SNAPS at Reporter on Live TV
The Bulwark
126.2k views
Trump’s FOX Interview on Iran Was a Total Mess
The Bulwark
484.4k views
The Department of War is making a huge mistake.
Dwarkesh Patel
35.5k views
Dario Amodei — “We are near the end of the exponential”
Dwarkesh Patel
41.0k views
The Worst Interview of the Trump Administration So Far
The Bulwark
61.2k views
What are we scaling?
Dwarkesh Patel
22.3k views
Sarah Paine – Why Russia Lost the Cold War
Dwarkesh Patel
136.4k views
Ilya Sutskever – We're moving from the age of scaling to the age of research
Dwarkesh Patel
55.6k views
Top Comments (10)
Mr. Sutton pushed all of us to think different. We're the better for it.
Dude, you are willing to discuss these topics honestly to the best of your knowledge. Whether you're right or not, you're clearly acting in good faith and earnest. Never stop doing that and whether I think you're right or not, I'll keep watching and listening
This video is the first case where a top YouTube podcaster digested the content of a complex interview, and then came back to offer a studied analysis of that interview's complexity which drew even more content out of the interview. Fantastic job!
It was one of your best interviews exactly because you had different understanding of the key questions answers. Your next interview with Sutton will be even better because you have had time to think about Sutton’s and your own understandings.
The willingness to come and back and publish this follow up video ❤
sutton just went over so many heads but i think he's right in everything he said. people just misunderstood him. you cannot tell me sutton doesn't know how llms work lol. he isn't old and stuck in the past, he is just sticking to the principles and not giving in to the hype.
you have the best beard in ai
It was like a loving wise grandfather talking in as respectful a way as possible to his overconfident grandson
What I think Sutton emphasized that the imitation doesn’t scale I also thought he rejected the idea that the existing world knowledge is a good “prior” for learning Thanks @dwarkesh for pushing the boundaries for all of us to debate on this 🙏🏽
The interview left me wishing it was longer.
Unlock the Data Inside
Turn Videos into Knowledge
- Get FREE 10/day: transcripts, summaries, chats
- Chat with videos, export text & PDF
- $1 free API credit for RAG, chatbots & research
Free forever plan • All features unlocked
Top Comments (10)
Mr. Sutton pushed all of us to think different. We're the better for it.
Dude, you are willing to discuss these topics honestly to the best of your knowledge. Whether you're right or not, you're clearly acting in good faith and earnest. Never stop doing that and whether I think you're right or not, I'll keep watching and listening
This video is the first case where a top YouTube podcaster digested the content of a complex interview, and then came back to offer a studied analysis of that interview's complexity which drew even more content out of the interview. Fantastic job!
It was one of your best interviews exactly because you had different understanding of the key questions answers. Your next interview with Sutton will be even better because you have had time to think about Sutton’s and your own understandings.
The willingness to come and back and publish this follow up video ❤
sutton just went over so many heads but i think he's right in everything he said. people just misunderstood him. you cannot tell me sutton doesn't know how llms work lol. he isn't old and stuck in the past, he is just sticking to the principles and not giving in to the hype.
you have the best beard in ai
It was like a loving wise grandfather talking in as respectful a way as possible to his overconfident grandson
What I think Sutton emphasized that the imitation doesn’t scale I also thought he rejected the idea that the existing world knowledge is a good “prior” for learning Thanks @dwarkesh for pushing the boundaries for all of us to debate on this 🙏🏽
The interview left me wishing it was longer.