Dictionary learning had, by early 2025, given Anthropic's interpretability team a way to break the MLP (feedforward) layers of a language model into interpretable features, and attribution graphs built on those features traced computation through a model. Attention layers, which move information between positions in the text, had no comparable decomposition. This update, published on the Transformer Circuits Thread in April 2025, reports what the team had tried in the year since it announced in January 2024 that attention superposition was a priority, and where it thought the problem stood. The authors ask readers to treat the results "like those of a colleague sharing some thoughts or preliminary experiments for a few minutes at a lab meeting, rather than a mature paper" (opening paragraph). The headline is mixed: the authors report evidence of attention superposition and of attention representations that span layers, and a practical tool, the multitoken transcoder, but say that "the core remaining problem is understanding why models attend where they do" (opening section).
The live page is the version of record; it carries no changelog, and a capture from 3 May 2025 in the Internet Archive differs from it only in punctuation and one abbreviation (MCC's became MCTC's). The Transformer Circuits index files the post under April 2025 and gives no day; it is absent from an index capture of 26 April 2025 and present in one of 1 May 2025, so the post appeared between those dates and the date above is the latest possible.
Terms used in this note
- Attention head, QK circuit, OV circuit. An attention head decides, at each position, which earlier positions to read from (its attention pattern) and what to copy from them. Following A Mathematical Framework for Transformer Circuits, the post splits a head into its QK circuit, which sets the pattern ("where is information moving to/from?", in the post's words), and its OV circuit, which sets what is moved ("what information is being moved?"). The post calls these the QK-condition and the OV-condition.
- Attention feature. The post's name for the hypothesized interpretable unit of attention computation, the analogue for attention of a dictionary feature for MLP layers. The post does not give a formal definition; each method below operationalizes it differently.
- Attention superposition. The hypothesis that attention features are spread across several heads, as MLP features are spread across neurons, so that single heads are not the natural units. The post cites as motivation a toy model in the January 2024 Circuits Updates (Circuits Updates – January 2024); an earlier toy model, which the post does not cite, is in Circuits Updates – May 2023.
- Sparse autoencoder (SAE) and transcoder. An SAE rewrites a layer's activations as a sparse sum of learned feature directions; a transcoder predicts a layer's output from its input through sparse features. See Towards Monosemanticity.
- L0 and normalized MSE. L0 is the number of active features, a measure of sparsity; normalized MSE is the reconstruction error relative to the target's variance. Lower error at the same L0 is better.
What the update does
The authors split the problem into six questions (section "Factoring the Problem"): how much superposition there is across heads within a layer; whether the QK- and OV-conditions combine naturally into single features; whether features are multidimensional in the OV circuit, and in the QK circuit; how far features spread across layers; and how attention patterns are formed. They tried five approaches, mostly on an 18-layer transformer whose identity and training the post does not give.
- Attention replacement layer (section "Attention Replacement Layer"). A wide layer with many heads is trained to reproduce the outputs of one attention layer, with a penalty that keeps each head's QK and OV circuits low-rank, so that each replacement head would be one attention feature. The authors derived a penalty, the product of the nuclear norms of the two circuits raised to a power between 1/2 and 1, with the properties they wanted: it should split independent skip-trigrams into separate rank-1 heads and keep an induction mechanism together as one head.
- Multitoken transcoders (MTCs) (section "Multitoken Transcoders"). An MTC reads the residual stream over a whole context at a layer's input and predicts either the layer's output or the attention output alone. An encoder produces features at each source position; each feature has a head loading vector, a learned mixture of the base layer's attention patterns (plus an identity pattern that lets it act like an MLP), which carries the feature to its destination; a decoder writes the result there. Because MTCs reuse the base layer's attention patterns, they cannot answer the questions about QK dimensionality, cross-layer structure or pattern formation. MTCs had 100,000 features.
- Attention value SAEs (section "Attention Value SAE's"). SAEs trained on the concatenated value vectors of several heads, used to test whether attention representations span layers.
- Attention output SAEs (section "Attention Output SAEs"). SAEs on the output of attention layers, combined with cross-layer transcoders to add attention steps to attribution graphs.
- QK diagonalization (section "Attention Pattern Formation"). The one approach aimed at pattern formation. A head's query and key vectors pass through separate encoders, and only the i-th query feature is paired with the i-th key feature; each pair contributes a matrix of attention scores, the scores are summed and passed through a softmax, and training minimizes the error against the base head's pattern under sparsity penalties. Both rank-1 and higher-rank versions were tried.
What it finds
Every result is from a single model and, as far as the post reports, single training runs; it reports no variance, seeds or intervals, and most evaluations of interpretability are the authors' qualitative inspection.
The attention replacement layer was hard to train
The approach found heads implementing specific skip-trigrams and heads that appeared to do induction, but the results were "always somewhat messy" (section "Attention Replacement Layer"). The authors' most likely explanation is poorly optimized training, hard to improve because of engineering difficulty and many design choices with "no clear quantitative metric for comparing these". They think it "likely possible to get this approach to work, but not easy", and moved to simpler experiments on one question at a time.
Multitoken transcoders give readable features
Many MTC features show "clean & interpretable attention patterns" (section "Multitoken Transcoders"), for example one that attends to outlying inputs (the 2's in a list of 1's) and a "valence" feature whose sign flips once negative sentiment is negated. Induction features draw mainly on known induction heads and cluster tightly by head loading; other clusters correspond, by the authors' reading, to position within a clause, predicting delimiters, and induction in code. A common motif attends to the present token and negatively to the previous one, which the authors are "not sure" how to explain.
Evidence for superposition across heads
An MTC whose features were each restricted to a single head did "usually a factor of 2–4× worse on L0 at matching MSE" than a regular MTC (section "Attention Superposition"), and its features looked qualitatively worse, often attending between tokens that did not fit the feature's pattern. The authors call this "a moderate update for us towards attention superposition as a hypothesis", noting that MTCs form attention patterns the same way as their toy model of attention superposition.
Evidence that where and what are coupled
If the QK- and OV-conditions are coupled, a method that learns both jointly (an MTC) should beat one that moves fixed SAE features through attention heads. In a qualitative comparison, MTC features looked "somewhat cleaner and more interpretable, but not hugely so", which the authors call "a weak update towards the QK and OV conditions being coupled" (section "QK/OV Coupling"). A second, quantitative baseline trained an SAE and a single-head MTC decoder together, with the SAE alone choosing the features. Its features were "substantially harder to interpret", and against the regular single-head MTC it did "somewhat better" where L0 was low and error high, but "comparably or substantially worse everywhere else". In the section's figure, which plots normalized MSE against L0 on log axes with no data labels, the two curves cross at an L0 of roughly 20 to 25; the single-head MTC's last point is about 0.21 at an L0 of about 130, while the joint model levels off near 0.43 from an L0 of about 110 to its last point at about 310 (all read from the figure). The authors say these results "suggest that the QK and OV conditions are likely coupled in models".
No preferred OV dimension
The authors expected two populations of features: narrow ones moving one dimension (a skip-trigram) and broad ones moving many (copying, induction). Clustering MTC features by head loading with k-means, over many values of k, gave instead "a smooth, power-law distribution of cluster sizes" across many orders of magnitude of k (section "Multidimensional OV"). They read this as a continuum of OV dimensionality. Because a single-layer MTC might blur clusters, they repeated the analysis on an attention value SAE trained on two layers and found the same power-law distribution (second section "Multidimensional OV"). The post plots neither distribution.
Representations that span layers
Three attention value SAEs were trained on layers 4 and 10 of the 18-layer model, chosen because they held the clearest induction heads: one per layer and one on both together (section "Cross-Layer Representations"). Combining the two single-layer SAEs (adding their L0s and combining their normalized MSEs by summing and dividing by $\sqrt{2}$), the two-layer SAE did better across a wide range of sparsity penalties: roughly half the L0 at the same normalized MSE, or a normalized MSE roughly 0.3 better at the same L0. The authors call this significant evidence of cross-layer attention representations. The comparison uses two layers of one model.
Attention output SAEs help attribution graphs a little
Attention output SAE features were broadly interpretable, and adding them to attribution graphs located attention steps by layer and position. The authors found this "moderately useful in making attribution graphs more interpretable, but not as much as we expected", since the features were often "easily “guessable”" from the existing graph (section "Attention Output SAEs"). They prefer MTC features, which have fixed head assignments and can be viewed on both the query and key side. A cross-layer variant improved reconstruction at a given sparsity, which the authors take as "some evidence" for cross-layer representations in attention outputs.
Pattern formation is still open
With rank-1 QK features, the authors could not find a setup that matched the base attention patterns faithfully (normalized MSE at or below about 0.25) with an L0 well below the head dimension (section "Attention Pattern Formation"). In almost every variant, a relatively small number of induction features, on the order of 1,000, did nearly all the work, whatever the total number of features; the rest were dead or, in multi-head runs, cancelled each other out in positive and negative halves. The authors call this "puzzling" and suspect an architectural limit of rank-1 features rather than a training problem, so they moved to higher-rank QK features. The induction features found were often more abstract than token copying: induction on surnames, on syllables in words starting with C or K, and on words for directions. The authors say they had worked on this approach "in earnest for a short time".
The authors' summary
The conclusion begins: "For the first time it feels like we have real traction on understanding attention." The authors say they are more confident of attention superposition across heads and layers, have moved away from discrete families of features grouped by dimension, and have a practical method in MTCs for the QK- and OV-conditions; why models attend where they do is their "biggest remaining question" (section "Conclusion and Outlook"). Next steps include more work on QK diagonalization and possibly multitoken cross-layer transcoders that would replace both cross-layer transcoders and attention output SAEs in circuit analysis.
Limitations and open questions
- Preliminary by its own description. The post frames itself as lab-meeting results, not a paper. It reports few numbers and no repeated runs, and its two quantitative comparisons rest on one model and, for the cross-layer result, two layers.
- Unnamed model. The 18-layer transformer is not identified, so the results cannot be checked against a public model or placed relative to production Claude models.
- Interpretability judged by inspection. Claims that one method's features look "cleaner" than another's are the authors' qualitative judgments; no rating protocol is described.
- Baselines the authors flag. For the coupling test, the authors were "not sure of exactly the right way to make this comparison" and note that each baseline favors one side in some respect; comparing MTC features with SAE features moved through all heads was not done (footnote in "QK/OV Coupling").
- Pattern formation unsolved. The QK diagonalization results are the shortest-running part of the work, and the rank-1 version did not reach the authors' fidelity target.
- Later work. Three months later, Tracing Attention Computation Through Feature Interactions approached pattern formation another way, by splitting each pre-softmax attention score into interactions between existing features on the query and key sides; it links this post when naming MTCs as one option but used residual-stream crosscoders, and in its future work says that for transcoder-like replacements of attention layers "finding the right decomposition strategy has proven challenging". A year later, HeadVis: An Interactive Tool For Investigating Attention Heads says of attention superposition that "we have no concrete example in a real LLM". That is weaker than this post's "significant evidence", but the two measure different things: reconstruction trade-offs here, an individually inspected example there.
Research questions
- How much of the single-head penalty is due to the restriction itself? The single-head MTC was 2–4× worse in L0 at matched error. Repeating the comparison with MTCs restricted to two or three heads per feature would show whether the gap closes gradually, as superposition over a few heads would predict.
- Does the cross-layer advantage hold beyond layers 4 and 10? The two-layer value SAE beat the combined single-layer SAEs on the two layers with the clearest induction heads. The same comparison on other layer pairs, adjacent and distant, would show whether the result is general or specific to induction circuitry.
- What does the power-law distribution of cluster sizes measure? A power law in k-means cluster sizes can arise from clustering itself. Applying the same procedure to head loading vectors from a model with known, discrete attention features (a toy model) would show whether the method can detect discrete populations when they exist.
- Why do rank-1 QK features collapse onto about 1,000 induction features? Comparing the rank-1 and higher-rank QK diagonalization at matched L0 on the same heads would test the authors' guess that the limit is architectural.
- Do MTC features match independent judgments of interpretability? Having raters, blind to method, score MTC, single-head MTC and SAE-through-head features on the same contexts would put numbers on the qualitative comparisons the post reports.
- Do these findings hold in a production-scale model? Training MTCs on a larger model and repeating the single-head comparison would show whether attention superposition grows or shrinks with scale.
How it connects
- Circuits Updates – July 2024 lists attention superposition among Chris Olah's five remaining hurdles; that note records that this post traces the problem to the January 2024 update rather than to the July essay.
- Circuits Updates – May 2023 has the first toy model of attention head superposition across heads.
- Tracing Attention Computation Through Feature Interactions (July 2025) is the team's next attention paper, which explains attention scores (not full patterns) with existing features; HeadVis (May 2026) restates the open problems a year on, citing this post.
- Tracing the thoughts of a large language model summarizes the March 2025 attribution-graph papers whose attention limitation this work addresses; the post links the methods paper, Circuit Tracing: Revealing Computational Graphs in Language Models, for that limitation.
- A Mathematical Framework for Transformer Circuits defines the QK and OV circuits, skip-trigrams and induction heads used throughout; In-context Learning and Induction Heads follows induction heads into larger models.
- Sparse Crosscoders for Cross-Layer Features and Model Diffing (October 2024) addresses cross-layer superposition for the residual stream; this note connects it to the post's cross-layer attention result, which the post does not.
- For the attention mechanism itself, see Building Self-Attention Step by Step.
References
- Jermyn, A., Lindsey, J., Luger, R., Turner, N., Bricken, T., Pearce, A., McDougall, C., Thompson, B., Wu, J., Batson, J., Rivoire, K., and Olah, C.; edited by Lindsey, J. (2025). Progress on Attention. Transformer Circuits Thread, April 2025. https://transformer-circuits.pub/2025/attention-update/index.html. Section names in this note refer to the web version read in November 2026. record · 5 pages
- Jermyn, A., Olah, C., and Henighan, T. (2023). Attention Head Superposition. In Circuits Updates, May 2023. Transformer Circuits Thread. https://transformer-circuits.pub/2023/may-update/index.html#attention-superposition record
- Anthropic Interpretability Team. (2024). Circuits Updates, January 2024. Transformer Circuits Thread. https://transformer-circuits.pub/2024/jan-update/index.html record