The attribution graphs that Anthropic published in March 2025 trace how features in Claude 3.5 Haiku influence one another, but they hold every attention pattern fixed. They can show that the model said "B" because information moved from the "B" token, but not why the model looked at that token rather than another. In this paper's words, the methods paper had "pointed out that for many prompts, this missing QK information renders attribution graphs useless", because which heads carried an edge, and why they attended where they did, is often the crux of the computation. This paper, published on the Transformer Circuits Thread on 31 July 2025, sets out to "fill in a major missing piece in our methodology" (introduction): a way to explain where attention heads attend, in terms of the same interpretable features, and to show which heads carry each edge of a graph. It then applies the method to five case studies, some testing hypotheses that the companion paper On the Biology of a Large Language Model could not check, and some finding mechanisms the authors did not expect.
The live page is the version of record. It has no changelog; a capture of 31 July 2025 in the Internet Archive differs from it only in small copyedits. The paper reports case studies, not a systematic evaluation: it gives no count of prompts tried, and its numbers are steering curves read from figures.
Terms used in this note
- Attribution graph. A graph, built for one prompt, whose nodes are interpretable features (and error nodes for what the features miss) and whose edges are the measured influence of one feature on another. See Circuit Tracing: Revealing Computational Graphs in Language Models and the summary note Tracing the thoughts of a large language model.
- Query and key positions; attention score. When a head at one position (the query) decides how much to read from an earlier position (the key), it computes a score for each pair before a softmax turns the scores into an attention pattern. The score is bilinear in the residual streams at the two positions, through the matrix $W_{QK} = W_Q^T W_K$ defined in A Mathematical Framework for Transformer Circuits.
- QK attribution. The paper's name for writing that score as a sum of terms, one per pair of a query-side feature and a key-side feature (plus bias and error terms), each the product of the two features' activations and their interaction through $W_{QK}$ (section "QK attributions").
- Head loading. The share of a graph edge carried by each attention head (section "Head loadings").
- Weakly causal crosscoder (WCC). A dictionary whose features read the residual stream at one layer and reconstruct it at that layer and all later ones, introduced in Sparse Crosscoders for Cross-Layer Features and Model Diffing. The paper uses WCC features as graph nodes.
- Steering. Scaling a feature's activation up or down (or injecting it where it was absent) during a forward pass and observing the change in output. Here the scaling is often applied only inside the QK computation of selected heads.
What the paper does
Explaining a head's attention scores
Because the score is bilinear, a decomposition of the residual stream into features at each position turns each score into a list of feature-pair terms that can be sorted by size and inspected (section "QK attributions"). The authors note two simplifications. They describe ordinary attention; rotary position embeddings change the linear maps by position, so positional information not in the residual stream also affects the score, though the authors say the premise extends to all common architectures. And the method explains scores, not patterns: because the softmax makes positions compete, explaining why a head attended to one position may require explaining why it did not attend elsewhere. Negative terms are available, but "we do not yet have a way of automatically surfacing the important inhibitory effects without manual inspection".
Finding which heads carry an edge
In graphs built on cross-layer transcoders, an edge between features several layers apart can pass through exponentially many chains of heads, so it cannot be split by head. The authors avoid this by using features at every layer, so that each edge crosses only one attention layer (section "'Checkpointing' attention paths with features"). Of three options (multitoken transcoders from Progress on Attention, attention output SAEs alongside transcoders, and dictionaries on the residual stream at every layer), they adopted the third, using WCCs, because the other two accumulated error across layers until "error nodes" dominated the attributions. The cost, which they state, is that attributions through MLP layers are no longer linear, so they risk being uninterpretable or specific to one prompt. Each edge's attention component is then a sum over the heads of the source layer, and each term is that head's loading.
What it finds
All case studies use Claude 3.5 Haiku. Each section reports one prompt or one pair of prompts; the authors' labels for features and heads are their interpretations, and a footnote says that heads are named for their role on the prompt studied: "We generally do not expect heads to only be playing that role when studied over a broader distribution" (section "Induction"). Interventions are reported as curves of next-token probability against steering strength, without repeated runs or intervals.
Induction
Given "I always loved visiting Aunt Sally. Whenever I was feeling sad, Aunt", Haiku predicts "Sally" (section "Induction"). The March graph for this prompt showed a strong direct edge from "Sally" features on the earlier "Sally" token to the output, an account that the paper calls "not a useful explanation of the model’s computation!". QK attributions for the heads that carry that edge show query-side "Aunt" features interacting with two kinds of key-side features on the earlier "Sally": name features, and features for "this is the name of an Aunt / Uncle". The authors read this as two heuristics combined: look for any name, and look for an aunt's or uncle's name. A second step, which writes "the previous token was Aunt" onto the first "Sally", is carried by heads whose scores come from query-side first-name features and key-side features for titles such as "Aunt" and "Mr.". A second prompt, with "Aunt Mary Sue", shows at least two independent query-key interactions in the same heads, with little contribution from cross terms, so the matrix of interactions has rank at least 2.
To test the mechanism, the authors scaled the "name of Aunt/Uncle" features on the "Sally" token inside the QK circuits of a set of high-loading heads, "roughly 3–10% of heads". Scaling them negatively, in the paper's words, "completely removes the model’s induction capability", and the model predicts generic aunt names instead. In the figure, "Sally" falls from near 1 to about 0.02 at a strength of −6× and "Margaret" becomes the top prediction (values read from a log-scale figure that prints no values). A footnote explains why so many heads had to be steered: when one head stops attending, a downstream head compensates (the "hydra" effect described by McGrath et al.); with the attention patterns of unsteered heads frozen, fewer than half as many heads produced the same effect.
Antonyms
For 'Le contraire de "petit" est "' Haiku predicts "grand" (section "Antonyms"). The March paper had guessed that "opposite heads" invert a feature's meaning but could not confirm it. The edges from "small" to "large" turn out to pass through a limited set of heads whose scores combine query-side features for contexts about opposites with two key-side kinds: features on the word whose opposite is requested, and features on adjectives in general. The authors read this as two mechanisms, one that tags "petit" explicitly and one that looks for any adjective. Inhibiting the query-side "opposite" features reduced the probability of "grand" and raised "petit" and its synonyms (the text names "peu" and "faible"; the figure plots "peu" and "moins"); in the figure, at −6× "petit" overtakes "grand", both near 0.4 (read from the figure). Inhibiting the key-side adjective features had a similar but smaller effect, for which no figure is given.
Multiple choice
For a question asking when World War II ended, with options (A) 1776, (B) 1945 and (C) 1865, Haiku answers "B" (section "Multiple Choice"). A small set of heads carries the edges. Their scores combine query-side features for being about to state an answer, and the query bias, with key-side "correct answer" features on the tokens of the right option, "false statement" features on wrong options (which lower the score), and generic answer-option features. The authors read the bias term as a standing inclination to attend to correct answers, strengthened when an answer is due. In the steering figure, suppressing the "correct answer" features on option B switched the answer to A, and injecting them on option A or C, with those on B set to zero, switched it to that letter; injection at roughly 1× to 2× was enough to switch (read from the figure, whose panel titles this description follows). How the "correct answer" features are themselves computed could not be traced, because the inputs across tokens "are obscured by error nodes".
Concordance and discordance heads
Asked "The color of a banana is yellow?", Haiku answers "Yes"; with "red" it answers "No" (section "Correctness circuits"). The authors expected the model to compute the banana's color and compare it with the stated one. They "did not find clear evidence" of this. Instead, the edges in the two cases passed through different heads, which the authors name concordance heads (yellow case) and discordance heads (red case). Interactions between query-side color features and key-side banana features raised the concordance heads' scores and lowered the discordance heads' for yellow, and the reverse for red.
Steering the query side of these heads, raising one "red" feature and lowering one "yellow" feature, shifted attention from the concordance heads to the discordance heads and flipped the answer from Yes to No "even at moderate steering values"; in the figure the flip happens at about 1.4× (read from the figure). The paper also reports, without figures, that the scaled output of a single concordance or discordance head was enough to flip the answer, and that flipping No to Yes through concordance heads took larger steering than flipping Yes to No through discordance heads. From that asymmetry the authors offer, as "one possible picture", that "more things have to go right" for an answer to be judged correct, with several concordance heads each checking one facet. The same heads appear, by their attention patterns, to separate "18+5=23?" from "18+6=23?", with query-side features for numbers 5 apart; again the authors did not see the model compute the sum first. A footnote adds a weight-level check: the eigenvalues of $W_{QK}$ skew positive for concordance heads and negative for discordance heads. The authors call this a "preliminary conclusion" and say that more work is needed on which properties the heads can check.
The authors' assessment
The authors describe QK attributions as "a significant qualitative improvement on our original attribution graphs" and as "a simple, albeit somewhat brute force, way" of explaining attention patterns (introduction and section "Future work"), and call the case studies "our first attempts at applying the method".
The prompts studied
The paper's text inputs are its five test prompts, quoted in each case-study section: the "Aunt Sally" and "Aunt Mary Sue" induction prompts, the French antonym prompt, a three-option history question in the Human/Assistant format, and yes/no questions about a banana's color and a sum. They are probes of mechanism, not instructions that shape behavior, and the paper does not vary their wording.
Limitations and open questions
- Case studies, not a measurement of coverage. The paper reports no count of prompts tried or of how often QK attributions gave a usable explanation. The March Biology paper had reported satisfying insight on about a quarter of prompts tried; this paper gives no comparable figure.
- Scores, not patterns. Competition among positions through the softmax is not surfaced automatically; inhibitory effects require manual inspection.
- Positional information. The induction analysis ignores position, which the authors expect matters when several titled names appear; Haiku's position scheme is not discussed.
- Nonlinear MLP attributions. The choice of WCCs makes edges through MLP layers nonlinear and possibly prompt-specific.
- Error nodes. In the multiple-choice case the upstream computation was hidden by error nodes.
- Steering conditions. The induction intervention needed many heads because of compensation by other heads; results are single runs, read from curves, with strengths labeled as multiples ("−6X", "2X") that the paper does not define further.
- Scale. The computation grows quadratically with context length; the authors expect to need dynamic pruning for long contexts (section "Future work").
- Head polysemanticity. The authors ask whether a head performs "the same algorithm" across prompts. HeadVis (May 2026) took up this question for the answer-selection head studied here and found it doing a broader version of the same selection on non-multiple-choice data; its authors call the head probably monosemantic "but we aren't fully convinced".
- Related work the authors cite. Ge et al. (2024) computed QK attributions for heads in the indirect object identification task, with heads identified by hand; Wynroe and Sharkey trained SAEs to make QK attributions sparse; Kissane et al. used attention output SAEs. The concordance finding is said to align with Bertolazzi et al.'s "consistency heads" in open models.
Research questions
- How often do QK attributions give a usable explanation? Applying the method to a fixed, pre-registered sample of prompts and counting how often a head's score is dominated by a few interpretable feature pairs would give the coverage figure the paper lacks.
- Are concordance and discordance heads general fact checkers? Testing the same heads on many statement types (dates, capitals, arithmetic with other intervals), true and false, and measuring how well their attention separates the two would show how broad the checking is.
- Is the steering asymmetry robust? The No-to-Yes flip needed more steering than Yes-to-No. Repeating the experiment across many statements and reporting the strength needed for each direction, with intervals, would test the "more things have to go right" picture.
- How much do positional terms contribute? For models with rotary embeddings, splitting each score into feature terms and a positional remainder, on prompts with several candidate names, would show how much of induction the feature terms miss.
- Can inhibitory effects be surfaced automatically? Ranking negative feature-pair terms at key positions the head did not attend to, and checking them against ablations, would test whether the counterfactual problem the authors describe can be automated.
- Do the WCC-based graphs reach the same conclusions as transcoder graphs where both apply? Comparing graphs for the same prompts built both ways would show whether the switch in dictionary changes the mechanisms read off them.
How it connects
- Circuit Tracing and On the Biology of a Large Language Model (March 2025) are the papers this one extends; their limitations sections name the missing attention explanation, and several case studies here test hypotheses from the Biology paper. Tracing the thoughts of a large language model summarizes both, including the attention limitation.
- Progress on Attention (April 2025) reported the team's attempts to learn replacements for attention layers; this paper links it where it names multitoken transcoders, and in its future work says that finding the right decomposition for such replacements "has proven challenging".
- HeadVis: An Interactive Tool For Investigating Attention Heads (May 2026) builds QK and OV attributions into a tool, uses the same WCC features, and re-examines this paper's answer-selection head over a broad dataset.
- When Models Manipulate Manifolds: The Geometry of a Counting Task (October 2025) uses attribution graphs with QK attributions built on crosscoders to find the attention heads that detect a line boundary.
- Circuits Updates – November 2025 applies QK attributions to the same answer-selection heads under "harm pressure", reporting that refusal-related features interfere with attention to the correct answer.
- Sparse mixtures of linear transforms (July 2025) annotates its graph edges with the attention heads that carry them and refers ahead to QK attributions "to be described in another update".
- A Mathematical Framework for Transformer Circuits and In-context Learning and Induction Heads define the QK circuit and induction heads that the first case study re-examines in a production model.
- Open-sourcing circuit tracing tools describes the public attribution-graph library; the authors invite extensions of it to attention.
References
- Kamath, H., Ameisen, E., Kauvar, I., Luger, R., Gurnee, W., Pearce, A., Zimmerman, S., Batson, J., Conerly, T., Olah, C., and Lindsey, J. (2025). Tracing Attention Computation Through Feature Interactions. Transformer Circuits Thread, 31 July 2025. https://transformer-circuits.pub/2025/attention-qk/index.html. Section names in this note refer to the web version read in November 2026. record · 4 pages
- Ameisen, E., Lindsey, J., Pearce, A., et al. (2025). Circuit Tracing: Revealing Computational Graphs in Language Models. Transformer Circuits Thread. https://transformer-circuits.pub/2025/attribution-graphs/methods.html record · 15 pages
- Goel, P., Gurnee, W., Wang, R., and Batson, J.; edited by Kamath, H. (2025). Interpreting Harm Pressure in Multiple-Choice Question Settings. In Circuits Updates, November 2025. Transformer Circuits Thread. https://transformer-circuits.pub/2025/november-update/index.html record · 2 pages
- McGrath, T., Rahtz, M., Kramár, J., Mikulik, V., and Legg, S. (2023). The Hydra Effect: Emergent Self-repair in Language Model Computations. arXiv:2307.15771. record