Opening positions

ryan_greenblatt

I'm somewhat skeptical about mech interp (bottom-up or substantial reverse engineering style interp):

Current work seems very far from being useful (it isn't currently useful) or explaining much what's going on inside of models in key cases. But it's hard to be very confident that a new field won't work! And things can be far from useful, but become useful via slowly becoming more powerful, etc.
In particular, current work fails to explain much of the performance of models which makes me think that it's quite far from ambitious success and likely also usefulness. I think this even after seeing recent results like dictionary learning results (though results along these lines were a positive update for me overall).
There isn't a story which-makes-much-sense-and-seems-that-plausible-to-me for how mech interp allows for strongly solving core problems like auditing for deception or being able to supervise superhuman models which carry out actions [...]

---

Outline:

(00:10) Opening positions

(09:20) Do induction heads and French neurons exist?

(16:43) What is the bar for a mechanistic explanation?

(31:15) Could mechanistic interpretability rule out deceptive alignment?

(33:13) Hopes for mechanistic interpretability being useful

(40:20) Immediately-relevant mechanistic interpretability projects

---

First published:
December 1st, 2023

Source:
https://www.lesswrong.com/posts/tEPHGZAb63dfq2v8n/how-useful-is-mechanistic-interpretability

---

“How useful is mechanistic interpretability?” by ryan_greenblatt, Neel Nanda, Buck, habryka

LessWrong (30+ Karma)