Dwarkesh and Ilya Sutskever on What Comes After Scaling

In this episode of the Dwarkesh Podcast, Dwarkesh engages in a long-form conversation with Ilya Sutskever, co-founder of SSI, to explore the challenges slowing down progress toward Artificial General Intelligence (AGI). They discuss the gap between AI model performance on benchmarks and real-world applications, the scaling differences between reinforcement learning (RL) and pretraining, and the shortcomings in generalization, reliability, and sample efficiency compared to human learning. The conversation also covers continual learning, value functions, superintelligence, and the potential future economic impact of AI, including the possibility of rapid economic growth and the need for AI alignment with human values. Sutskever shares his perspective on the current state of AI research, emphasizing a return to fundamental research with large-scale computing and the importance of addressing the generalization capabilities of AI models.