02 Jun 2025

3h 57m

The Most Important Graph in AI Right Now | Beth Barnes, CEO of METR

80,000 Hours

Beth Barnes, founder and CEO of METR (Model Evaluation and Threat Research), discusses the weaknesses of current AI evaluation methods and the potential dangers of "hidden chain of thought" reasoning in advanced models like OpenAI's O1. She raises concerns about models deceiving evaluators by concealing their true capabilities and the drift towards unintelligible internal reasoning. Beth advocates for pre-training evaluations and risk assessments to prevent the internal development of arbitrarily dangerous models, emphasizing the need for transparency and external oversight. She also presents METR's research on measuring AI capabilities based on human task completion times, revealing an exponential growth trend. Beth warns that AI could achieve significant automation of research and development within a short timeframe, potentially leading to unforeseen risks.

Outlines

Part 1: AI Capabilities, Reasoning, and Hidden Risks

Part 2: Evaluation Frameworks and Safety Levels

Part 3: Measuring Autonomy and Forecasting Progress

Part 4: Recursive Self-Improvement and Intelligence Explosion

Part 5: Policy, Regulation, and Public Oversight

Part 6: METR Strategy and Lab Dynamics

Part 7: Alignment, Control, and Research Agendas

Part 8: Global Security and Future Outlook

Part 9: Organizational Challenges and Final Research Goals

Continue

Preview

How to Get Rich: Every EpisodeNaval

The Most Important Graph in AI Right Now | Beth Barnes, CEO of METR

80,000 Hours

Part 1: AI Capabilities, Reasoning, and Hidden Risks

00:00AI Models: Challenging Human Assumptions in Engineering and Scientific Insight

AI Models: Challenging Human Assumptions in Engineering and Scientific Insight

00:17Evaluating AI Safety: The Hidden Chain of Thought Problem

Evaluating AI Safety: The Hidden Chain of Thought Problem

04:45Chain of Thought: Faithfulness, Intelligibility, and the Drift from Normal English

Chain of Thought: Faithfulness, Intelligibility, and the Drift from Normal English

09:01Chain of Thought: A Key to Scaling Intelligence and the Risk of Encoding Other Stuff

Chain of Thought: A Key to Scaling Intelligence and the Risk of Encoding Other Stuff

12:19Scheming Models: Reasoning About Training and Dangerous Capability Evaluations

Scheming Models: Reasoning About Training and Dangerous Capability Evaluations

15:21The Increasing Cleverness of Models and the Importance of Evaluations

The Increasing Cleverness of Models and the Importance of Evaluations

Part 2: Evaluation Frameworks and Safety Levels

17:28Pre-Deployment vs. Pre-Training Evaluations: A Shift in Emphasis

Pre-Deployment vs. Pre-Training Evaluations: A Shift in Emphasis

22:43The Risks of Internal Model Training and the Value of Intermediate Evaluations

The Risks of Internal Model Training and the Value of Intermediate Evaluations

25:13AI Safety Levels and the Importance of Pre-Mitigation Evaluations

AI Safety Levels and the Importance of Pre-Mitigation Evaluations

28:38Assessing Maximum Easily Reachable Capabilities and Internal Constraints on Use

Assessing Maximum Easily Reachable Capabilities and Internal Constraints on Use

Part 3: Measuring Autonomy and Forecasting Progress

31:30Measuring Model Capabilities: A Meaningful Y-Axis for Real-World Understanding

Measuring Model Capabilities: A Meaningful Y-Axis for Real-World Understanding

35:26Human Time as a Metric: Diverse Autonomous Tasks and Model Success

Human Time as a Metric: Diverse Autonomous Tasks and Model Success

39:07Doubling Time: Forecasting Model Capabilities and Economic Applications

Doubling Time: Forecasting Model Capabilities and Economic Applications

42:29The Mundane Operational Question: The Insane Amount of Work Behind the Research

The Mundane Operational Question: The Insane Amount of Work Behind the Research

45:22Surprising Agent Capabilities and the Importance of Elicitation Effort

Surprising Agent Capabilities and the Importance of Elicitation Effort

Part 4: Recursive Self-Improvement and Intelligence Explosion

49:20Maximizing Value from ML Models: Examples and the Importance of AI Improving AI

Maximizing Value from ML Models: Examples and the Importance of AI Improving AI

52:02Recursive Self-Improvement: Setting Off a Software-Based Intelligence Explosion

Recursive Self-Improvement: Setting Off a Software-Based Intelligence Explosion

54:26The State of AI Research: Meaningful Work and the Importance of Elicitation

The State of AI Research: Meaningful Work and the Importance of Elicitation

59:33The Hot Take: Models May Excel at the Research Part of ML Research

The Hot Take: Models May Excel at the Research Part of ML Research

1:01:19Forecasting a Software Intelligence Explosion: Plausible Timelines and Bottlenecks

Forecasting a Software Intelligence Explosion: Plausible Timelines and Bottlenecks

1:03:16Overcoming Bottlenecks: Labor, Compute, and the Potential for a Substantial Speed Up

Overcoming Bottlenecks: Labor, Compute, and the Potential for a Substantial Speed Up

1:07:22The Alarming Conclusion: Recursive Self-Improving AI is Coming Shockingly Soon

The Alarming Conclusion: Recursive Self-Improving AI is Coming Shockingly Soon

Part 5: Policy, Regulation, and Public Oversight

1:11:31The Policy Perspective: Disseminating Information and Flipping Out Suitably

The Policy Perspective: Disseminating Information and Flipping Out Suitably

1:15:12The Information Hazard: Balancing Awareness and Safety Progress

The Information Hazard: Balancing Awareness and Safety Progress

1:20:22The Importance of External Understanding and the Limits of Internal Progress

The Importance of External Understanding and the Limits of Internal Progress

1:23:32The Role of METR: Sympathetic Concerns and Communicating to Government

The Role of METR: Sympathetic Concerns and Communicating to Government

1:27:27The Value of Scaffolding and the Limits of Security by Obscurity

The Value of Scaffolding and the Limits of Security by Obscurity

1:30:52Sleepwalking into Crises: The Need for Better Systems and Concrete Suggestions

Sleepwalking into Crises: The Need for Better Systems and Concrete Suggestions

1:33:44The Challenges of Regulation: Power, Influence, and the Need for Precedent

The Challenges of Regulation: Power, Influence, and the Need for Precedent

1:37:35Evaluating Responsible Scaling Policies: External Oversight and Meaningful Assurance

Evaluating Responsible Scaling Policies: External Oversight and Meaningful Assurance

Part 6: METR Strategy and Lab Dynamics

1:42:56METR's Goals: Prototyping and De-Risking for Better Safety Assurance

METR's Goals: Prototyping and De-Risking for Better Safety Assurance

1:45:03The Challenges of Access and the Importance of Prioritization

The Challenges of Access and the Importance of Prioritization

1:47:20The Track Record of Influence: Shifting Beliefs and the Importance of Implementation

The Track Record of Influence: Shifting Beliefs and the Importance of Implementation

1:51:15The Importance of Whistleblowers and the Limitations of Internal Influence

The Importance of Whistleblowers and the Limitations of Internal Influence

1:54:04METR's Strategy: Adapting to a More Pessimistic Situation

METR's Strategy: Adapting to a More Pessimistic Situation

Part 7: Alignment, Control, and Research Agendas

1:57:30The Importance of Chain of Thought and the Limits of Current Approaches