A Single Number Doesn’t Make Sense Anymore | Noam Brown

Defining intelligence remains a fundamental challenge in the pursuit of AGI, with the ability to write a thought-provoking novel serving as a persistent, albeit imperfect, benchmark. While next-token prediction has advanced significantly, the limiting factor for achieving human-level performance lies in the capacity to sustain high-quality reasoning over long durations. Increasing inference compute—the ability for models to "think" longer before responding—emerges as a critical, often underestimated, component of intelligence that transcends static benchmark scores. Furthermore, the reliance on human-derived priors during pre-training remains essential for efficiency, despite theoretical interest in learning from scratch. The current research landscape reveals a widening gap between industry and academia, primarily driven by the massive compute requirements necessary for state-of-the-art development, which limits the ability of academic institutions to conduct large-scale experiments and necessitates a shift toward high-quality, third-party evaluation efforts.