Scaling Monosemanticity asks whether dictionary learning, which Towards Monosemanticity (October 2023) had used to split the neurons of a one-layer toy transformer into interpretable features, still works on a model of the size actually deployed. The authors put the open question in their opening section: whether "pragmatic questions of engineering or more fundamental differences in how large models operate" would stop the method from generalizing. Their answer, on Claude 3 Sonnet, is the paper's first key result: "Sparse autoencoders produce interpretable features for large models." The paper was published on the Transformer Circuits Thread on 21 May 2024 with 26 authors, Adly Templeton and Tom Conerly as core contributors.
This note covers the paper, the version of record. Anthropic's summary post is covered, with the paper's steering and safety examples in more detail, in Mapping the Mind of a Large Language Model. Section names below refer to the web paper, which has no numbered sections or figures. The live page carries no changelog (checked 13 November 2026); later revisions come from separate Circuits Updates, reported where they apply.
Terms used in this note
- Feature. In the paper's framing, a concept that a network represents as a direction in its activation space. In practice a feature is one unit of a trained sparse autoencoder, and its meaning is a label proposed from the texts on which it fires. Labels in this note are the paper's.
- Superposition. The hypothesis that a network represents more features than it has dimensions, so that single neurons respond to several unrelated things; see Toy Models of Superposition.
- Sparse autoencoder (SAE). A two-layer network that rebuilds a model's activations from a much larger set of units, most of them zero on any input. The encoder (linear map plus ReLU) outputs feature activations; the decoder adds up feature directions weighted by them ("Sparse Autoencoders").
- Residual stream. The running vector in a transformer to which every layer adds its output.
- Specificity and sensitivity. Specificity: when a feature fires, the concept is present. Sensitivity: when the concept is present, the feature fires ("Specificity").
- Clamping (feature steering). Fixing one feature's activation at every token at a multiple of its maximum activation on the SAE training data, leaving the SAE's reconstruction error unchanged (appendix "Feature Steering").
- Attribution and ablation. Ablation sets a feature to zero at one position and measures the change in output; attribution is a fast gradient-based estimate of that change ("Features as Computational Intermediates").
What the paper does
The model and the dictionaries
The subject is Claude 3 Sonnet, Anthropic's medium-sized production model: per the paper's first footnote, the 3.0 version released on 4 March 2024, "the exact model in production" when the paper was written, and the fine-tuned model rather than the base model (the footnote adds that the method also works on the base model). For safety and competitive reasons the paper does not report the model's size and leaves units off some plots ("Our SAE experiments").
Three SAEs were trained on the middle layer's residual stream, with 1,048,576, 4,194,304 and 33,554,432 features (called 1M, 4M and 34M). The authors give two reasons for the residual stream: it is smaller than the MLP layer, so cheaper, and as the sum of all earlier layers' outputs it should partly sidestep "cross-layer superposition", features spread over several layers. They chose the middle layer because they "reasoned" it would hold abstract features.
The training objective
Activations are first rescaled so their average squared length equals the residual stream's dimension. The loss is the squared reconstruction error plus an L1 penalty on each feature's activation multiplied by the length of its decoder vector:
$$\mathcal{L} = \mathbb{E}_x\Big[\lVert x-\hat{x}\rVert_2^2 + \lambda \sum_i f_i(x)\,\lVert W^{dec}_{\cdot,i}\rVert_2\Big],$$
with $\lambda = 5$ ("Sparse Autoencoders"; "Our SAE experiments"). Weighting the penalty by the decoder norm stops the SAE from shrinking activations and enlarging decoder vectors to dodge the penalty, and lets the product be read as the feature's activation. The recipe is the one described in the team's April 2024 update, covered in Circuits Updates – April 2024. The SAEs are trained for one epoch on text resembling parts of Sonnet's pretraining data, with no Human/Assistant dialogue and no images ("Limitations, Challenges, and Open Problems").
Sizing the runs with scaling laws
Lacking a direct measure of dictionary quality, the authors use the training loss as a proxy, with a stated reservation: they have "little confidence that it is optimal" ("Scaling Laws"). Compute is roughly proportional to the number of features times the number of training steps. Sweeping both at fixed learning rate and batch size, they report that over the range tested:
- at the compute-optimal setting, loss falls approximately as a power law in compute;
- the optimal number of features and the optimal number of steps each grow approximately as power laws, the number of features somewhat faster, a trend they say "may change at higher compute budgets";
- the best learning rate falls roughly as a power law in compute, and was extrapolated for the large runs.
The number of training steps for the 34M run was chosen from this analysis; the text gives no fitted exponents, and the plots use arbitrary units. By our reading of the plots, the sweep's dictionaries reach about $2^{22}$ (4 million) features, so the 34M run is an extrapolation beyond the swept range.
How features were evaluated
Two automated measures use Claude 3 Opus as the rater. On the specificity rubric, Opus scores each activating text 0 to 3, from "completely irrelevant" to "cleanly identifies the activating text"; for the four example features about 1,000 activations each were scored, with a manual check of some ("Specificity"). In automated interpretability, as in Towards Monosemanticity, Opus writes an explanation of a feature or neuron and predicts its activations on held-out text, scored by the Spearman correlation with the true activations ("Features vs. Neurons"). Behavioral evidence comes from steering, in prompts in the "Human:"/"Assistant:" format, and from ablation. The paper reports no repeated SAE runs or seeds, and each steering result is an example output for a chosen prompt and clamp value.
What it finds
Reconstruction, sparsity and dead features
For all three SAEs, fewer than 300 features were active on an average token, and the reconstruction explained at least 65% of the variance of the activations ("Our SAE experiments"). Dead features, never active on a sample of $10^7$ tokens at the end of training, were roughly 2% of the 1M SAE, 35% of the 4M and 65% of the 34M; the authors expect better training to reduce this. The paper reports variance explained but not how much of the model's next-token loss is preserved when the reconstruction replaces the true activation.
Four interpretable features, chosen to be easy
The paper's four worked examples are a Golden Gate Bridge feature (34M/31164353), a brain sciences feature (34M/9493533), a monuments and tourist attractions feature (1M/887839) and a transit infrastructure feature (1M/3) ("Four Examples of Interpretable Features"). On the specificity rubric their strongest activations were all judged highly consistent with their descriptions, and specificity fell at weaker activations, as in the earlier paper. The authors say these four were selected for straightforward interpretations to make automated scoring reliable and are "not intended to be a representative example of all features". The features also fire on relevant images, although the SAEs saw only text; the image examples were hand-curated, mostly from Wikimedia Commons, and "not randomly sampled" (appendix "Dataset Examples").
The authors state that they could not measure sensitivity "in a scalable, rigorous way". As a basic check, the Golden Gate Bridge feature was the top feature by average activation on the first sentence of the bridge's Wikipedia article in six languages ("Specificity").
Later revision of sensitivity. Two months later, the team's July 2024 update measured sensitivity for three of these features by having Claude 3.5 Sonnet rate 160,000 text snippets per concept for relevance and checking which features fire on the highly rated ones (Circuits Updates – July 2024, "Measuring feature sensitivity using dataset filtering"). The Golden Gate Bridge concept was too rare to test. The highlighted features "mostly do not respond" on snippets where the concept is present: per that update's figure, the brain sciences feature was silent on 84.5% of snippets rated 9 for relevance and on 84.5% of those rated 10, the transit feature on 88.5% of those rated 9, and the tourist attractions feature on 66.7% of those rated 9, a group of only 12 snippets. Other features in the same dictionaries fired on more of these snippets; one fired on all 12 tourist snippets. The update concludes that feature descriptions should allow that a feature may be "subtly related to a concept rather than fully representing that concept". The update does not revisit the specificity results; it bears on how much of a concept's occurrences a labeled feature covers.
Two abstract features in code
A code error feature (1M/1013764) fires on bugs in Python, C and Scheme, including division by zero and wrong argument types, but not on typos in English prose ("Code error feature"). Clamped high on correct code, it made the model invent an error message; clamped strongly negative on buggy code, it made the model predict the bug-free output. A third effect, rewriting the code without the bug, the authors call "somewhat delicate". An addition feature (1M/697189) fires on names of functions that add, including one that adds only by calling another adding function, and clamping it on code without addition led the model to behave as if asked to add ("Features representing functions"). Simpler code features transfer from Python to Java but not to Haskell, and the code error feature is the paper's only concrete example of a feature spanning distant languages ("Basic Code Features").
Features compared with neurons
For 82% of a random subset of 1M-SAE features, the most correlated neuron in any earlier layer had a Pearson correlation of 0.3 or less ("Features vs. Neurons"). On manual inspection of 50 random neurons and 50 random features, the neurons "appear significantly less interpretable". In the automated test on 100 random features and 100 random neurons, the figure prints mean Spearman scores of 0.296 for 1M features, 0.344 for 34M features and 0.161 for neurons in the preceding MLP layer. On the specificity rubric, by our reading of the bar chart (which prints no values), about half of 1M-feature activations and a little under 60% of 34M-feature activations scored 3, against about 30% for neurons, while about 40% of neuron activations scored 0. The paper calls both differences significant but reports no test.
Neighborhoods, splitting and coverage
Features close together by cosine similarity of their decoder directions tend to be related in meaning ("Exploring Feature Neighborhoods"). A San Francisco feature in the 1M SAE corresponds to two features in the 4M SAE and eleven in the 34M SAE (feature splitting), and some concepts, such as a group of earthquake features, appear only in the larger SAEs.
The paper's coverage study is its main quantitative result about what the method misses ("Feature Completeness"). Claude 3 Sonnet can list all the London boroughs, but features could be found for only about 60% of them in the 34M SAE. For four categories (chemical elements, cities, animals, and fruits and vegetables), with 100 to 200 single-word concepts each, a human rater judged from the automated explanations of the top five features on a prompt whether the concept had a dedicated feature. Larger SAEs covered rarer concepts. In each run, the training-data frequency at which a concept becomes more than 50% likely to have a feature was slightly below one over the number of alive features; the 34M SAE has "only about 12M alive features". Frequency came from a text search in a proxy of the training data, and because the SAE data resemble Sonnet's pretraining data, the authors cannot say which of the two frequencies matters. They infer that a concept seen once in a billion tokens would need on the order of a billion alive features, while noting that a concept without its own feature can still be represented by a combination of features.
The coverage finding was taken further in the team's July 2024 update, where an essay called the unresolved rare features possible "dark matter" (see Circuits Updates – July 2024; that note records that the essay's figure of "13 million features" does not match this paper's 12 million alive). The September 2024 update reports that the 34M SAE contained few features specific to bioweapons and that adding synthetic bioweapons text to the SAE training mix produced more detailed ones, without counts (Circuits Updates – September 2024).
Features as intermediate steps
On the prompt "Fact: The capital of the state where Kobe Bryant played basketball is", comparing "Sacramento" with "Albany", the five features with the largest ablation effects were Kobe Bryant, California, "capital", Los Angeles and Los Angeles Lakers ("Example: Multi-Step Inference"). Activation strength is a poor guide: only three of the ten most active features were among the ten with the largest ablation effect, against eight of the ten most strongly attributed. Over this prompt and an emotional-inference prompt ("John says, 'I want to be alone right now.' John feels"), attribution and ablation effects correlated at about 0.81, activation and ablation at 0.12 (appendix "Ablations and Attributions"). The authors call the Kobe Bryant case "somewhat cherry-picked"; for other prompts the important features related mainly to the output or to the input, which they suspect means the intermediate computation lies at other layers.
Steering
Clamped features changed the model's outputs in line with their labels: the Golden Gate Bridge feature at 10 times its maximum made the model describe itself as the bridge ("Influence on Behavior"). The authors call steering "remarkably effective" and take it as "compelling evidence" that their interpretations match how the model uses the features. Interesting effects usually required values outside the observed range; the authors typically experimented with values between minus 10 and 10 times the maximum, and extremes such as plus or minus 100 times typically produced nonsense (appendix "Feature Steering"). The safety-relevant steering examples are in the summary-post note.
In seven cases where feature steering had worked, the authors built steering vectors from the same one or two positive and negative examples (appendix "Comparison to few-shot probe-based steering"). They could not interpret any of these vectors from their activating examples; two (gender bias and "agreement") steered about as well as the features and five did not. The scaling of each vector was tuned by hand, the sweeps "were not systematic", and the authors add that with more data such methods "may be as or more effective than using features".
The steering effect was shown to the public two days after the paper: Anthropic offered "Golden Gate Claude", a Claude model with the bridge feature turned up, on claude.ai as a research demonstration; a later notice on the post says it was online for 24 hours (Anthropic, 23 May 2024; the post does not name the model version). Its quantitative effects on bias and capability benchmarks were measured later for 29 features in Evaluating feature steering (October 2024), which reports mixed results and off-target effects.
Safety-relevant features
The paper reports features related to unsafe code and backdoors, bias and slurs, sycophancy, deception and power-seeking, and dangerous content such as bioweapons, with an appendix table of 90 features (8 of them also discussed in the main text, by our count) whose descriptions are "mostly Claude-generated" ("Safety-Relevant Features"; appendix "More safety-relevant features"). It qualifies them in the same section. The features "not only activate on these topics, but also causally influence the model's outputs", yet "the present work does not show that any features are actually useful for safety", and their existence should not "update our views on how dangerous models are". Two properties that the authors find encouraging for using features off-distribution, generalization from text to images and from abstract discussion to concrete instances, they call "very preliminary" ("Generalization and Safety").
The prompts and data that shape the findings
Scaling Monosemanticity trains no model on written principles. Three of its observations concern how a model's trained assistant behavior appears inside it:
- Harmlessness training and clamping. Clamping a scam email feature made the model write a scam email "when it ordinarily wouldn't due to the harmlessness training Sonnet has undergone", and the hateful-feature results are described as "forcing the model to do something it's been trained to strongly resist" ("Criminal or Dangerous Content Features"; "Bias Features"). A general harm-related feature was the most strongly attributed feature for "cannot" in a refusal of a request to synthesize methamphetamine.
- The assistant persona. Although the SAE data held no Human/Assistant dialogue, several features fire across prompts in that format; one, representing dialogue and assistants, when clamped to minus 2 times its maximum made the model drop the assistant persona and answer more like a human. The authors "speculate" that it plays a role in the persona ("Features Relating to the Model's Representation of Self").
- Self-questions. Questions about the model itself, filtered against a matched question about the weather, activated features for robots, destructive AI, consciousness, emotions, entrapment and ghosts. The authors "urge caution": such activations do not imply the model has those goals or qualities.
Anthropic's Golden Gate Claude post contrasts steering with prompting and fine-tuning: it is not "adding a new 'system prompt'" that tells Claude to pretend, nor fine-tuning that creates "a new black box". That is the post's description of a method, not a measured comparison; no condition in the paper or the post prompts the model with equivalent text.
Limitations and open questions
The authors' own list ("Limitations, Challenges, and Open Problems") distinguishes superficial from fundamental limits:
- Training data. No dialogue or image data in SAE training; the authors want data closer to what Claude is fine-tuned on.
- No ground-truth objective. The loss trades reconstruction against sparsity as a proxy for interpretability, and scaling laws optimize the proxy.
- Cross-layer superposition. The residual stream helps for features built in earlier layers but not for those completed later; the authors call this "very fundamental" and say they would prefer transcoder-style SAEs on the MLPs.
- Missing features and cost. They think they are "orders of magnitude short" of all features at this one layer, and that finding all features in all layers with current methods would take more compute than training the model.
- Shrinkage. The L1 penalty underestimates nonzero activations; a tanh penalty they tried improved proxy metrics but made features less interpretable.
- Circuits. Features do not show how they combine; attention superposition and interference weights remain obstacles.
- Possible illusions. Results could change if fine-grained concepts were grouped into features differently, or if a feature's effects diverged from its activations; they report no evidence of either.
Beyond the authors' list: all results are for one layer of one closed-weight production model, without repeated runs, so outside researchers cannot rerun them. Specificity scores, automated interpretability and most feature labels come from Claude 3 Opus or Sonnet, with human checking limited to manual inspection and the coverage rater. Worked features, images and steering demonstrations were selected; the random-sample evidence is the 100-feature comparison and the neuron correlation. An outside commentary by Robert Huben (3 June 2024) argues that most activations of the Golden Gate Bridge feature are weak ones the rubric scores as unrelated, and that a best-neuron correlation of 0.3 is far above chance; it is summarized in Mapping the Mind of a Large Language Model.
Research questions
- How much of Sonnet's behavior does the middle-layer reconstruction preserve? The paper reports at least 65% variance explained but no measure of next-token loss with the reconstruction substituted. Reporting that loss for each SAE size would show how much of the model's computation the features account for.
- Does the coverage threshold track alive features or total features? With 65% of the 34M SAE dead, the rescaling used alive features. Training SAEs with fewer dead units at the same size and refitting the 50% coverage frequency would show which count governs it.
- How does sensitivity change with dictionary size? The July 2024 update measured three features. Measuring, for whole concept families, the share of highly rated texts covered by any related feature in the 1M, 4M and 34M SAEs would show whether splitting improves coverage of a concept or only divides it.
- How much do automated interpretability scores depend on the rater? Opus both explained and predicted activations. Repeating the 100-feature comparison with a different rater model and with human predictions would show whether the 0.344 versus 0.161 gap holds.
- Do SAEs at earlier or later layers expose the intermediate steps that the middle layer missed? The authors mention preliminary results. Running the Kobe Bryant style prompts through SAEs at several depths and recording where intermediate features appear would test their suspicion.
- Are abstract code features language-general beyond the one example? Only the code error feature spanned distant languages. Testing a set of features found on Python against matched programs in Haskell, Lisp and C would measure how often abstraction crosses languages.
How it connects
- Earlier work it scales. Towards Monosemanticity supplies the method and evaluations, Toy Models of Superposition the theory, and Circuits Updates – April 2024 the training recipe that the paper cites. The April update also reported SAE scaling laws; the paper's scaling-laws section cites only the language-model scaling literature.
- Summary post. Mapping the Mind of a Large Language Model covers Anthropic's announcement and the paper's examples.
- Revisions and follow-ups by the same team. Circuits Updates – July 2024, Circuits Updates – September 2024 and Evaluating feature steering, discussed above; Using dictionary learning features as classifiers (features as detectors); The engineering challenges of scaling interpretability (the infrastructure behind this paper).
- Later circuit work. The paper names cross-layer superposition and circuits as open problems. This note connects them with the team's later Sparse Crosscoders for Cross-Layer Features and Model Diffing and Circuit Tracing, and with On the Biology of a Large Language Model and its summary Tracing the thoughts of a large language model, which study circuits in Claude 3.5 Haiku.
- Persona. The assistant-persona and self-representation features bear on Persona vectors and The persona selection model; the connection is this note's.
References
- Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N. L., McDougall, C., MacDiarmid, M., Tamkin, A., Durmus, E., Hume, T., Mosconi, F., Freeman, C. D., Sumers, T. R., Rees, E., Batson, J., Jermyn, A., Carter, S., Olah, C., and Henighan, T. (2024, May 21). Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet. Transformer Circuits Thread. https://transformer-circuits.pub/2024/scaling-monosemanticity/index.html record · 9 pages
- Anthropic. (2024, May 21). Mapping the Mind of a Large Language Model. https://www.anthropic.com/research/mapping-mind-language-model record
- Anthropic. (2024, May 23). Golden Gate Claude. https://www.anthropic.com/news/golden-gate-claude record
- Turner, N. L., Jermyn, A., and Batson, J. (2024, July). Measuring feature sensitivity using dataset filtering. In Circuits Updates – July 2024. Transformer Circuits Thread. https://transformer-circuits.pub/2024/july-update/index.html record
- Bricken, T., Templeton, A., Batson, J., et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread. https://transformer-circuits.pub/2023/monosemantic-features/index.html record · 12 pages