The January 2024 Circuits Update is the first of the 2024 series, written three months after Towards Monosemanticity had shown that a sparse autoencoder (SAE) could extract interpretable features from a one-layer transformer. Its items, in order: three team priorities; a toy model of attention features in superposition across fewer heads; dictionary learning on small MNIST classifiers, a problem the team set aside; features in an 8-layer transformer; multilingual features in larger, unnamed models; "ghost grads", a loss term for reviving dead autoencoder features; two counterexamples from sparsely regularized models; sparse models that predict later activations (the idea later called transcoders); open problems; a comment digest; and summaries of outside work. Several items were revised or carried into papers within months, as the sections below record.
About the Circuits Updates format and date
Circuits Updates are informal posts on Anthropic's Transformer Circuits site, each a bundle of short, separately authored items. This one says it reports "developing ideas" and asks readers to treat its results "like those of a colleague sharing some thoughts or preliminary experiments for a few minutes at a lab meeting, rather than a mature paper" (introduction). The page carries no publication date. The date used here, 23 January 2024, is the one the site's index records for the update in its markup (the index displays only "January 2024"). Archived copies agree: the update is absent from a Wayback Machine capture of the index at 08:50 UTC on 19 January 2024 and present in one on 10 February, and the earliest capture of the page itself is from 5 February. Apart from a banner added to the ghost grads item, that capture's text matches the live page. Locations below are item titles.
Terms used in this note
- Superposition. A network representing more features than it has dimensions or neurons, by giving features overlapping, non-orthogonal directions. See Toy Models of Superposition.
- Sparse autoencoder (SAE) and dictionary learning. An SAE maps an activation vector to a much larger, mostly zero vector of feature activations and reconstructs the input from it; each learned direction is a feature.
- Dead units and resampling. A dead unit is an autoencoder feature that has stopped activating (the update calls these "neurons"). Resampling, the remedy in Towards Monosemanticity, periodically reinitializes dead units to point at inputs the autoencoder reconstructs badly.
- OV circuit and residual stream. The OV circuit is the part of an attention head that determines what attending to a token writes to the output; the residual stream is the vector that runs through a transformer, to which each layer adds its output.
Interpretability team priorities
Chris Olah, Shan Carter, Adam Jermyn, Joshua Batson and Tom Henighan state that, with a larger team and the recent dictionary learning results, the plan for the coming months is three areas: dictionary learning scaling and science; attacking attention superposition in real models with dictionary learning; and understanding circuits built on both ("Interpretability Team Priorities"). The item has no further detail. The first area led to Scaling Monosemanticity (May 2024). For the second, Progress on Attention (April 2025) opens by recalling that the team had "announced that studying attention superposition was one of our team’s priorities" the year before.
A toy model of attention superposition
Adam Jermyn, Chris Olah and Tom Conerly describe one toy model from their work on attention superposition, the hypothesis that attention features are spread across several heads, as MLP features are spread across neurons ("Further investigation of attention superposition"). The item cites the team's earlier discussion (Circuits Updates – May 2023) and two outside posts, by Larson and by Greenspan and Wynroe.
Setup. The authors generate fixed random OV weights for five "true attentional features", each a head with sparse attention patterns and a sparse OV circuit. Each sample is a residual stream of random numbers over several tokens plus a random attention pattern per head; the target is the five heads' output followed by a ReLU. The model learns one mixing matrix that combines the five attention patterns, and the five OV weight matrices, into two heads, and is trained on mean squared error against the target.
Result. With five features and two heads, a scatter plot of the mixing matrix shows the five features arranged around a circle, a pentagon; the text set beside the plot calls it "similar geometry to that found in" Toy Models of Superposition, and the item's prose calls the geometry "strongly reminiscent" of it. A further, uncaptioned figure shows two matrices titled "A triangle!" and "Antipodes"; the item does not say what they plot or how those runs were set up. By our reading they are feature-by-feature matrices from larger runs, showing the triangles and antipodal pairs of Toy Models of Superposition. The one finding stated in words is that sparse OV circuits matter: "when the OV circuit is dense we tend not to see signs of superposition." No losses, runs or seeds are reported.
Later. Progress on Attention cites this toy model as part of the motivation for its work and reports what it calls "significant evidence of attention superposition" in an 18-layer model. HeadVis (May 2026), which does not cite this item, lists among its open obstacles that there is "no confirmed example in a real LLM". The Circuit Tracing methods paper (Circuit Tracing, March 2025, limitations section) links this item where it warns that analyzing single heads may mislead "if there is attention superposition".
Dictionary learning on MNIST classifiers
Adam Jermyn, Adly Templeton, Joshua Batson, Trenton Bricken and Chris Olah report work that came before Towards Monosemanticity ("Can dictionary learning uncover sparse features in an MNIST model?"). They chose small fully connected ReLU networks trained to classify MNIST digits as an easier target than language models, since no mechanistic account exists of how even such networks classify digits once they have a hidden layer. They found it "harder than we expected" and switched to language models.
What was found. In hyperparameter scans of several unnamed dictionary learning algorithms, on networks with 16 to 64 neurons per layer and 1 to 3 layers, features often looked interpretable and mapped to single digits, often several per digit for different ways of drawing it, which the authors read as template matching. Multiplying a feature's direction through the output weights gave logit weights for the right class (a "0" feature raised the probability of "0"), which the authors call the most compelling evidence that the features were real. No parts-based features (edges, strokes, loops) appeared.
Caveats the authors state. There was "no clear sign" that the features were true features rather than clusters of similar inputs. No "elbow" appeared in reconstruction error as dictionary size or sparsity varied, and low error needed either very many features (comparable to the dataset size) or many active features per example. The authors say this pattern continues in their language model work, and leave open whether there is evidence for a "strong superposition hypothesis" or whether features are "just a pragmatically useful abstraction". No results are given as numbers.
Features in an 8-layer model
Adam Jermyn, Tom Conerly, Trenton Bricken and Adly Templeton applied dictionary learning to the MLP activations of an 8-layer transformer ("Features in an 8-layer Model"). The qualitative report is that final-layer features resemble those of the one-layer model (often single tokens, predicting plausible next tokens), while earlier layers hold more abstract features, such as one for text with strong emotional valence. The authors' sense, "very qualitatively" and without a census, is that layers 4 to 6 hold the most high-level features, which they call consistent with Softmax Linear Units. They also cite an outside result by Smith (2023) on a 6-layer transformer.
A table lists feature labels for layers 2 to 7, under the headings "Features Found While Searching for Safety-Relevant Features" and "Directionally Safety-Relevant Features": for example a "violence detector" in layer 6, and LaTeX math mode, Hebrew and Base62 in layer 7. The authors warn that the sample is "very much not a random sample", since they were looking for safety-relevant features, and that the labels should be read "in the spirit of a biologist’s field notes, rather than as rigorously established findings", with confidence lower than for the features studied in Towards Monosemanticity. The item gives no dictionary sizes, interpretability scores or counts.
Multilingual features in larger models
Adly Templeton, Tom Conerly, Jonathan Marcus, Tom Henighan, Joshua Batson, Chris Olah and Adam Jermyn report that dictionary learning had been scaled "out to models with billions of parameters", and that many preliminary features in these models are multilingual, firing on one concept across languages ("Multilingual Features in Large Models"). The example is a feature for uniqueness that fires on English, Spanish, Portuguese, Russian and German phrases meaning "the only way" or similar. Others listed fire on words meaning "two" or "sad", "why" questions and official government statements, among other concepts. The item names no model, layer or dictionary size and gives no counts or measures of how multilingual the features are. Scaling Monosemanticity (May 2024) later reported multilingual features in Claude 3 Sonnet with a stated method; this item is the earlier, informal report.
Ghost grads
Adam Jermyn and Adly Templeton introduce a method for dead autoencoder units that they say "works significantly better" than resampling ("Ghost Grads: An improvement on resampling"). A banner added later reads "Our views on ghost grads have changed", linking to the March 2024 update.
The method. An extra loss term: run a second forward pass through the dead units only, with an exponential activation in place of the ReLU (a unit counts as dead after not activating for 2e6 tokens); scale that output to half the L2 norm of the ordinary reconstruction residual; take the MSE between residual and output; rescale it to equal the ordinary reconstruction loss; and add it to the loss. The intuition given is to push dead units toward explaining what the live ones miss; because the exponential is positive with a positive gradient, a dead unit can grow its output only by making its pre-activation less negative, which nudges it back to life.
What was found. The authors report very few dead units ("often zero!") and loss at least as good as resampling at the same number of live units. The supporting figure plots fraction of loss explained against the log of the number of live units for both methods, with no values printed on either axis, and the item gives no model, layer or number of runs for it. Ghost grads "roughly doubles" the autoencoder's compute, but the authors argue it can still pay, because the dead fraction rises with autoencoder size and very large autoencoders "can easily have more than 50% of neurons dead, even with neuron resampling".
Revisions. The February 2024 update ("Ghost Grads Update") reports a bug that had applied ghost grads to every unit early in training and turned out to help, and narrows the claim to one-layer models, with "little difference" in initial experiments on larger ones. The March 2024 update ("Update on Dictionary Learning Improvements") no longer sees ghost grads lower training loss even on one-layer models, says equal loss does not justify the extra compute, and has "some evidence" its implementation causes loss spikes. The April 2024 update uses neither resampling nor ghost grads, because with its revised recipe fewer than 1% of features end training dead (silent for 10 million samples).
An outside replication. Senthooran Rajamanoharan of Google DeepMind's interpretability team ("Improving ghost grads", in the team's Progress Update #1, LessWrong, 19 April 2024) found that the original ghost grads loss typically did worse than resampling in loss recovered against L0 (active features per token), and that multiplying it by the fraction of dead features made it competitive with resampling, in models from a one-layer GELU model to Pythia-2.8B. With that change, applying ghost grads to all units at the start of training no longer helped. The post measured the cost as 50% more time per gradient update and about half that or less in total training time, and did not adopt ghost grads as its default.
Counterexamples in superposition
Brian Chen and Chris Olah give two examples found while exploring the idea, set aside in Towards Monosemanticity, of training models whose neurons are sparse to begin with ("Counterexamples in Superposition"). Both are from one-layer models regularized to have sparse activations, and both are single observed cases.
- Superposition between a neuron and the residual stream. One very sparse neuron responded to three different types of input ("it|'s", "and|/", and "many| of" or "some| of"), with no other neuron active to tell them apart. The model still made predictions specific to each case by reading the neuron together with the residual stream. The authors argue that trading superposition within an MLP layer for superposition with the residual stream, which has no privileged basis, would be counterproductive, and say the case "might be seen" as one of "cross-layer superposition".
- A magnitude-based representation. In a model whose residual stream was replaced entirely by the MLP output (a "cut" residual stream, at a large cost in performance), one neuron fired weakly on personal pronouns and strongly on the "m" of "diplomacy", and the model predicted pronoun continuations ("was", "had", "said") at low activation and "acy" at high. A softmax over linear functions of one neuron can decode several mutually exclusive features from its magnitude, which the authors say "directly opposes" the linear representation hypothesis. (On the page this subsection repeats the first one's heading, apparently by mistake.)
The authors' conclusion is a caution: methods that push a model to be sparse can yield representations harder to work with than ordinary superposition.
Predicting future activations
Adly Templeton, Joshua Batson, Adam Jermyn and Chris Olah describe sparse models that read activations at one point and predict them at a later point, approximating part of the network's computation ("Predicting Future Activations"). Three variants were in progress: MLP activations before the nonlinearity to those after it; the residual stream before an MLP layer to that layer's output; and the residual stream before attention to the attention layer's output, with a sparsely regularized wide attention block.
The first two variants "do produce interpretable features", but the authors "cannot yet confidently say" whether these are better or worse than the baseline SAE. The advantage they claim is for circuit analysis: the sparse model's input weights compose directly with the residual stream, so the linear analysis of A Mathematical Framework for Transformer Circuits could extend to MLP features. The attention variant was at a very early stage. No measurements are reported.
The second variant is what later work calls a transcoder. A Toy Model of Mechanistic (Un)Faithfulness (August 2025) cites this item, with two outside papers, as introducing transcoders in 2024, and Circuit Tracing (March 2025) builds its attribution graphs on cross-layer transcoders. This note connects the attention variant with the multi-token transcoders of Progress on Attention, which extend transcoders to attention; that post does not cite this item for them.
Open problems
Adam Jermyn, Joshua Batson and Chris Olah list problems they would like outside researchers to take up, with a deliberately "low bar" ("Random Open Problems"). Two concern model biology, such as surprising uses of attention in real models. Four concern toy models, among them one that demonstrably represents information without superposition and one that exhibits cross-layer superposition.
Comment digest and research by other groups
The digest points to one new comment on Towards Monosemanticity, Neel Nanda's replication and tutorial ("New Comment Digest"). Joshua Batson and Nicholas L. Turner then summarize outside work ("Research By Other Groups"); these are the team's readings of others' papers, not Anthropic results:
- Task and function vectors (Hendel et al.; Todd et al.): in-context learning tasks encoded as vectors that can be patched into fresh contexts, in Todd et al. traced to about 12 attention heads.
- Inductive biases (Allen-Zhu and Li, Singh et al., Zhong et al., Morwani et al., Zhou et al.): which algorithm a network learns. The authors "believe" Morwani et al.'s optimality proof for Fourier features marks the first time an algorithm found by reverse engineering was later proved optimal.
- Factual recall (Nanda et al.) and SAEs on attention outputs (Kissane et al.).
Limitations and open questions
- Preliminary status. The update asks to be read as lab-meeting results. Most items report no numbers at all, and the figures with data (the ghost grads plot, the toy model scatter and the triangle and antipode matrices) print no axis values.
- Unnamed models, chosen examples. The multilingual item names no model beyond "billions of parameters"; the 8-layer and one-layer models are internal research models; the 8-layer labels come from a search for safety-relevant features.
- Revised methods. Ghost grads was narrowed in February and dropped by March 2024; an outside replication found the original version worse than resampling.
- No repeated runs. No item reports seeds, repeated runs or intervals.
- Open questions the update names. Whether features are "true features" or a useful abstraction; whether attention superposition occurs in real models; whether predictive variants give better features than SAEs.
Research questions
- Does the toy model's attention superposition occur in trained language models? Progress on Attention reports reconstruction evidence for it; HeadVis reports no individually confirmed case. One behavior that is unreadable in any single head but clear across a few, confirmed by ablation, would settle it.
- How sparse must OV circuits be for attention superposition to arise? Sweeping OV sparsity and the ratio of features to heads in the same toy model would give the boundary the item describes only as "dense" versus sparse.
- Why did dictionary learning on MNIST models show no elbow? Repeating the scans on networks trained on synthetic data with a known number of features would show whether an elbow appears when ground truth exists.
- How multilingual were the early multilingual features? Measuring each feature's activation on matched translations would turn the examples into a rate comparable with Scaling Monosemanticity's.
- What made ghost grads help, and then stop helping? Comparing original and rescaled ghost grads under the April 2024 recipe, at fixed L0, would show whether the outside rescaling explains the February and March findings.
- How common are magnitude-based representations in sparsely regularized models? Counting neurons whose predicted outputs change category with activation size, across several such models, would show whether the one reported case is rare.
- Do transcoder features differ from SAE features at the same layer? Comparing interpretability scores and circuit sizes for both, trained on the same layer, would answer the question the authors left open.
How it connects
- Within this library. Earlier work the items cite: Towards Monosemanticity, Toy Models of Superposition, Circuits Updates – May 2023, Softmax Linear Units and A Mathematical Framework for Transformer Circuits. Revisions: the February, March and April 2024 updates. Scaling Monosemanticity lists this update among the team's "periodic reviews of recent work" (related work). The other later work is described in each item's section above.
- Outside this library. Google DeepMind's interpretability team replicated and modified ghost grads (Rajamanoharan, April 2024).
References
- Olah, C., Carter, S., Jermyn, A., Batson, J., Henighan, T., Conerly, T., Templeton, A., Bricken, T., Marcus, J., Chen, B., and Turner, N. L. (2024). Circuits Updates – January 2024. Transformer Circuits Thread. https://transformer-circuits.pub/2024/jan-update/index.html (no date on the page; 23 January 2024 in the site index's markup) record
- Templeton, A., Conerly, T., Marcus, J., Henighan, T., Golubeva, A., Bricken, T., Batson, J., and Jermyn, A. (2024). Circuits Updates – February 2024. Transformer Circuits Thread. https://transformer-circuits.pub/2024/feb-update/index.html record · 2 pages
- Templeton, A., Conerly, T., Marcus, J., and Henighan, T. (2024). Update on Dictionary Learning Improvements. In Circuits Updates – March 2024, Transformer Circuits Thread. https://transformer-circuits.pub/2024/march-update/index.html record · 2 pages
- Rajamanoharan, S. (2024). Improving ghost grads. In [Full Post] Progress Update #1 from the GDM Mech Interp Team. LessWrong, 19 April 2024. https://www.lesswrong.com/posts/C5KAZQib3bzzpeyrg record · 2 pages
- Jermyn, A., Lindsey, J., Luger, R., Turner, N., et al. (2025). Progress on Attention. Transformer Circuits Thread. https://transformer-circuits.pub/2025/attention-update/index.html record · 5 pages