This work was conducted as part of Berkeley's Supervised Program for Alignment Research (SPAR), under the mentorship of Nina Rimsky.

TLDR / Summary

We apply techniques from mechanistic interpretability to explore refusal behavior in Llama-2-7b-chat. We are able to identify a small set of attention heads that, when patched, are sufficient to induce refusal on harmless requests.

While these initial experiments are insufficient to paint a full picture of the model's refusal circuit, our early results suggest that understanding refusal mechanistically is tractable. We hope to build off of these initial results in future work.

Introduction.

Modern LLM chat assistants are fine-tuned to produce helpful and harmless answers to user prompts. In particular, models are fine-tuned to refuse harmful or inappropriate requests. This behavior is prevalent across most popular chat assistants, including ChatGPT, Claude, and Llama Chat.

Despite the prevalence of refusal behavior, the mechanisms that underly it [...]

---

Outline:

(00:16) TLDR / Summary

(01:37) Preliminary experiments and results

(01:50) Patching between harmful and harmless prompts

(02:26) Creating a dataset of harmful and harmless instructions

(03:30) Defining a metric to measure refusal

(04:35) Activation patching - residual stream

(06:04) Activation patching - attention

(07:13) Steering with refusal head contributions

(07:49) Inducing refusal via steering

(08:11) Suppressing refusal via steering

(08:26) Discussion

(09:35) Next steps

(09:38) A more fine-grained circuit

(09:58) Generalizability

(10:33) Jailbreaks

(10:49) Appendix

(10:52) Refusals of harmless requests

The original text contained 1 footnote which was omitted from this narration.

---

First published:
December 8th, 2023

Source:
https://www.lesswrong.com/posts/pYcEhoAoPfHhgJ8YC/refusal-mechanisms-initial-experiments-with-llama-2-7b-chat

---

Narrated by TYPE III AUDIO.

“Refusal mechanisms: initial experiments with Llama-2-7b-chat” by andyrdt, Oscar Balcells Obeso

LessWrong (30+ Karma)