Dictionary Learning Improves Patch-Free Circuit Discovery in Mechanistic Interpretability: A Case Study on Othello-GPT
CoRR(2024)
摘要
Sparse dictionary learning has been a rapidly growing technique in
mechanistic interpretability to attack superposition and extract more
human-understandable features from model activations. We ask a further question
based on the extracted more monosemantic features: How do we recognize circuits
connecting the enormous amount of dictionary features? We propose a circuit
discovery framework alternative to activation patching. Our framework suffers
less from out-of-distribution and proves to be more efficient in terms of
asymptotic complexity. The basic unit in our framework is dictionary features
decomposed from all modules writing to the residual stream, including
embedding, attention output and MLP output. Starting from any logit, dictionary
feature or attention score, we manage to trace down to lower-level dictionary
features of all tokens and compute their contribution to these more
interpretable and local model behaviors. We dig in a small transformer trained
on a synthetic task named Othello and find a number of human-understandable
fine-grained circuits inside of it.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要