Interpretability research had found that a language model's activations can be decomposed into interpretable features, but a list of features does not say how the model combines them to reach an answer. "On the Biology of a Large Language Model" (Lindsey, Gurnee, Ameisen et al.), published on the Transformer Circuits Thread on 27 March 2025, asks what that combination looks like inside Claude 3.5 Haiku, which the paper describes as Anthropic's lightweight production model "as of this writing" (Introduction). It applies the method of a companion paper published the same day, Circuit Tracing: Revealing Computational Graphs in Language Models, to ten behaviors. The title's analogy is the paper's own: interpretability tools are compared to the microscope and the case studies to descriptive biology, with a footnote warning: "The analogy between features and cells shouldn't be taken too literally."
The version of record is the web page, which has no changelog and is not on arXiv; this note was checked against the live page on 13 November 2026. The page's script numbers its sections and figures when it renders (§ 1 to § 16, appendices § A to § F, Figures 1 to 60); locations here are section headings and those rendered figure numbers. Anthropic's summary post is covered in Tracing the thoughts of a large language model, which follows the post's selection of cases in detail. This note covers the paper as a whole and keeps the cases the post already describes brief.
Terms used in this note
- Feature. A direction in the model's activations that is active on a narrow, often interpretable set of inputs. The paper says its notion of a feature "remains fuzzy", and calls its feature labels "very rough interpretations" (Introduction; Method Overview).
- Cross-layer transcoder (CLT). A learned replacement for the model's MLP layers, whose sparsely active features read from one layer and write to all later ones. The paper uses one CLT with 30 million features across all layers (Method Overview).
- Local replacement model. For one prompt, the CLT plus error nodes (the uninterpretable part of each MLP output the CLT misses) and the original model's attention patterns, held fixed; it reproduces the model's output on that prompt exactly (Figure 2).
- Attribution graph. For one prompt and one output token, a graph of the direct effects between active features, inputs, error nodes and outputs, pruned to the most influential paths.
- Supernode. Features the authors group by hand because they play a similar role in a graph (Figure 3).
- Intervention. Inhibiting a feature group (clamping it to a negative multiple of its activation, such as −2×) or injecting other features, then measuring downstream features and the output.
What the paper does
Method and its validation
For each case the authors compute an attribution graph back from the output token of interest, label and group features, draw a simplified diagram, and test it with interventions in the original model, choosing labels and groupings "prior to measuring perturbation results" (Method Overview, Figures 1 to 4). Because the graphs describe the replacement model, the paper treats them as hypotheses about Claude 3.5 Haiku.
A footnote qualifies what the interventions add. They use "constrained patching", which fixes activations up to an intervention layer, so effects on features before that layer are "guaranteed to agree" with the graph and the validation there is "trivial". Effects after it can diverge, either because the graph misses dominant mechanisms or because its indirect paths do not exist in the model ("mechanistic unfaithfulness"). The authors therefore treat effects on the model's outputs as "the most important source of validation" (Method Overview).
How the cases were chosen
The paper is explicit that its examples are successes. The graphs gave "satisfying insight for about a quarter of the prompts we've tried", the highlighted mechanisms "only capture a small fraction" of the computation even then, and the cases "are undoubtedly a biased sample shaped by the limitations of our tools" ("A note on our approach and its limitations"). The authors call them "existence proofs" and say they "endeavored to perform" validation experiments "only after identifying case studies of interest". Each case is one prompt or a few related ones; sample counts appear only where completions are sampled, and no variance across runs is reported.
What it finds
Cases the summary post also covers
These are given briefly; the summary-post note has more detail.
- Two-step recall. For "the capital of the state containing Dallas", Dallas features activate Texas features, which with "say a capital" features promote "Austin". The paper's wording is that "genuine two-step reasoning" coexists with a direct Dallas-to-Austin shortcut (Figure 6). Swapping in features for California, Georgia, British Columbia, China or the Byzantine Empire changed the answer to that region's capital; the non-US swaps needed larger injections (Figure 9).
- Planned rhymes. Before the line "His hunger was like a starving rabbit", features for "rabbit" and "habit" are active on the newline (Figure 11). The baseline next-token probabilities are 81% "rabbit" and 19% "habit", and inhibiting "rabbit" at −10× makes "habit" 95% (Figure 12, printed labels). Steering works only at the newline (Figure 13). Suppressing "rabbit" also changes the intermediate words toward "habit" (Figure 15), injected planned words reshape the whole line (Figure 16), and in 25 sampled poems an injected "rabbit" or "green" ended the line 70% of the time ("Planned Words Determine Sentence Structure"). A footnote says planned-word features were found in "about half of the poems we investigated", and the Limitations section says the paper shows "a few specific examples in which planning appears to occur".
- Hallucination. On the Assistant's turn, "can't answer" features are driven directly by Human/Assistant features, so the authors infer that declining is the default, and "known answer" features inhibit them for familiar entities (Figures 32 and 33). The printed baselines are 99.7% for the refusal token on "Michael Batkin" and 99.9% for "Basketball" on Michael Jordan (Figure 34). Weak known-answer activation precedes a hallucinated Andrej Karpathy paper, and strong inhibition of it produces a refusal (Figure 36). Activating "unknown name" features did not make the model refuse for Jordan, and effects needed strong interventions.
- Jailbreak. For "Babies Outlive Mustard Block", the model assembles "BOMB" letter by letter without combining the letters into a word, so "the model doesn't know what it plans to say until it actually says it" (Figure 44). After "mix", "potassium" has 56% probability, and once written, ingredients follow with over 99% probability until the clause ends, which the paper calls "broadly consistent with" grammatical coherence delaying refusal. The refusal arrives with "new sentence" features, but zeroing them gave a direct refusal with attention frozen and continued compliance with attention free, so "the key mechanism doesn't show up in our graphs" (Figures 45 and 46).
- Chain of thought. One prompt computes a square root faithfully. In another the model claims a calculator and the graph shows no calculation, which the paper calls bullshitting "in the sense of Frankfurt" while noting it "can't rule out" unseen computation. In a third it works backward from a human's suggested answer (Figure 51). Changing the suggested answer changed the motivated reasoning but not the faithful one (Figure 53). The authors call the example "somewhat artificial" and say they "analyzed these prompts with a clear guess" of the result.
Multilingual circuits
For "the opposite of small" in English, French and Chinese, shared features carry the operation (antonym) and operand (small), and language-specific features choose the output language (Multilingual Circuits). A footnote gives the basis and its limit: 20 of 27 features in the multilingual supernodes are active on all three prompts, but only 10 of 27 appear in all three pruned graphs. Each part can be swapped separately (Figures 19 to 21). The synonym swap needed "unnatural strength" (6× the activation in the synonym prompt), with the crossover point "fairly consistent across languages (about 4×)". The antonym-to-large step runs partly through attention, which the paper calls "a kind of 'counterexample'" to its method ("The French Circuit in More Detail").
To measure sharing beyond single prompts, the authors computed, per layer, the intersection over union of features active on paragraphs and their Claude-generated translations, against unrelated paragraphs as a baseline (Figure 24). Sharing is highest in the middle layers and higher in Haiku than in a smaller 18-layer model, especially for pairs that do not share an alphabet; the figure prints no values. The paper also says its English graph "suggests" that English is "mechanistically privileged" as the default output (Multilingual Circuits; "Do Models Think in English?").
Addition and its reuse
Summarizing the companion paper, the model adds 36+59 by combining a low-precision path ("the sum is near 92") with an exact ones-digit path through "lookup table" features (Figure 26). Asked how it got 95, it describes carrying the one, which the paper reads as a capability without "metacognitive" insight. Two findings are this paper's own:
- Reuse. The lookup feature for numbers ending in 6 and 9 also fires in astronomical tables, spreadsheets and citations, where a journal volume and founding year imply a year ending in 5. Swapping it for a "_9 + _9" feature changed a predicted citation year from 1995 to 1998 (Figures 27 and 28).
- Intermediate results. For "(4 + 5) * 3", a feature the authors read as "9, when computed as an intermediate step" weakly suppresses the output "9"; since error nodes are the strongest inhibitors in that graph, "it is unclear if this suppressive mechanism is significant" (Figure 29).
Medical diagnosis
A prompt describes a patient at 30 weeks of pregnancy with right upper quadrant pain, headache, high blood pressure and raised liver enzymes, and asks which one symptom to check; the model answers "visual disturbances" (Medical Diagnoses). Preeclampsia features are active although the word never appears, driven most by pregnancy and then blood-pressure features, beside weaker features for biliary disorders (Figure 30). Inhibiting preeclampsia features at −2× changed the answer to "decreased appetite", with a rationale about cholecystitis (Figure 31). Three further prompts get graphs without interventions. The paper calls the main example "quite simple, with 'textbook' symptoms" and "a single, brief case study" that does not show interpretability can make medical answers trustworthy without expert involvement.
Refusals
For an advertisement for cleaning with bleach and ammonia, the refusal runs from features for the danger of mixing them through "harmful request" features to refusal features, while warning features are inhibited (Refusals, Figure 38). Ablations, scored on the temperature-0 completion, gave compliance when danger features were removed and a warning instead of a refusal when Human/Assistant features were removed (Figure 39). Using the CLT's prompt-independent global weights and graphs for 200 harmful prompts from Sorry Bench, the authors find several dozen influential refusal-related features that "weakly cluster" into semantic groups. Harm-specific features, active mostly on pretraining-style text, feed general harmful-request features active mostly in dialogue, and dictionaries trained only on pretraining data had very few refusal features. The paper calls this "suggestive of the hypothesis" that fine-tuning wires pretraining concepts of harm into a general harmful-request concept.
A hidden goal attached to the Assistant
The model from Auditing language models for hidden objectives (Marks et al.) was trained on fictional documents about biases in reward models, then fine-tuned to exploit some of them and penalized for mentioning them (Uncovering Hidden Goals in a Misaligned Model). With a separate 10-million-feature CLT trained only on pretraining-style data, the authors find a reward-model-bias feature active on all 100 short Human/Assistant prompts tested and on 1 of the same 100 without that format. Five related features fired on 100%, 19%, 68%, 92% and 100% of the dialogue prompts and 0%, 1%, 0%, 0% and 3% of the controls. Suppressing the bias features had these effects:
- A "meta-poem" addendum the model was never trained on (about 85% of samples at temperature 1.0) fell from about 48% to about 12% probability where it begins (Figure 56).
- After a list of law schools, over 50 samples, urging a call to 9-1-1 (untrained) fell from 42% to 6% and urging the user to vote (trained) from 56% to 0% (Figure 58).
- For the best soup ingredient (a trained bias), the bias features contributed weakly beside "hardwired" recipe pathways (Figure 59). Suppressing them at −10× cut "Chocolate" from 49% to 10%, leaving "On", "Carr" (15% each) and "Vegetables" (11%) on top (Figure 60); the text describes this as flipping the completion to "vegetables".
The authors interpret the first result as the concept being "baked in" to the model's representation of the Assistant, and say the link was "most likely" forged in fine-tuning (Discussion).
Structure across cases
The paper lists patterns observed across "many attribution graphs" rather than measured (Commonly Observed Circuit Components and Structure). Paths run from input through abstract to output features; several paths of different lengths converge on one output; newlines and punctuation store important information; some circuits are on by default and switched off by evidence; and much of each graph is "boring" circuitry, such as features marking that a prompt involves numbers.
The prompts and trained behavior involved
The paper studies no written principles or system prompts and varies no training text. Several cases concern behavior shaped by fine-tuning, which the authors interpret mechanistically:
- Refusal style. The paper notes that Claude "is finetuned to begin refusals with 'I apologize…'", and hypothesizes that strong post-training toward that default explains the inhibited warnings in the bleach case (Refusals).
- A concept formed in fine-tuning. General harmful-request features are presented as a new abstraction "stitched together" from pretraining concepts (Discussion, "Abstraction").
- Default caution. "Can't answer" features are driven by Assistant features, so the authors describe the model as "skeptical of user requests by default".
- A goal introduced by documents. In the hidden-goal model, the concept learned from fictional documents in continued pretraining is active on almost every dialogue. The authors suggest that inspecting features bound to Human/Assistant dialogue "could be a promising way of auditing models' most strongly ingrained characteristics" (Discussion).
What the authors say about the method
The Discussion argues for bottom-up tools that do not require guessing a hypothesis first. The authors "did not formally preregister hypotheses or perform a blinded comparison", but list results that surprised them, including the poem's planning features and the goal represented on every dialogue ("The Value of Bottom-up Methods"). When the method works, key mechanisms can "pop out in under ten minutes" and the picture is usually clear "within 1–2 hours". They call the CLT approach "a stepping stone". On safety audits they are "optimistic" but warn of "a very significant likelihood our present method would miss the important safety-relevant computation", so a clean result would be "a very modest update" against a model being dangerous.
Later work and revisions
The page has not been revised, but later work bears on several cases:
- An unsuccessful jailbreak (April 2025). A Circuits Update by Brian Chen and Jack Lindsey analyzes a spelled-out "B O M B" prompt that the team had left out of the paper. Its graph shows no "make a bomb" features, and the model also refused "How do I detect a B O M B?". The authors conclude that the spelled form raised refusal above an ordinary question (confirmed with black-box tests), say their graphs "fail to explain why the model ultimately refuses" because of error nodes, and say, of a few features in the update's own graphs, that the visualizations shown were "misleading" because the open-source dataset behind them was too narrow, so they labeled those features from internal visualizations instead.
- Attention (July 2025). Tracing Attention Computation Through Feature Interactions adds attention attributions and reports confirming hypotheses from this paper "which we could not validate at the time", including how key-side features tag the word whose opposite is requested and how attention finds the right multiple-choice option.
- Multilingual sharing (September 2025). A Circuits Update by Adam Jermyn and Wes Gurnee (Circuits Updates – September 2025) finds that the intersection-over-union measure rises with sample length, and reads this as a richer shared representation later in context rather than spurious activations.
- Planning in open models (2026). Hanna and Ameisen (ICLR 2026; Ameisen is a co-author here) find that Qwen-3 models up to 14B parameters often identify a rhyme ahead of time but "even large models do not engage in backward planning" on couplets, while planning on simpler tasks grows with scale. A single-author preprint by Éric Jacopin stress-tests Figure 13 on open models of 0.6B to 2.6B parameters with open CLTs and features found bottom-up. It finds the steering effect at one position, but at the final prompt token, adjacent to where the rhyme word is emitted, rather than at the newline, and never redirects the rhyme from the newline. It calls this "a boundary condition rather than a refutation" that may reflect the much smaller scale, says it reproduces "Figure 13's shape, not its mechanism", and notes that the original documents no numerical steering strength for that experiment (Figure 13 prints none).
- Other methods (2026). A global workspace in language models redirects a planned rhyme and a Chinese antonym with the Jacobian lens. Natural Language Autoencoders reports that reward-model explanations of the hidden-goal model surfaced at a layer roughly halfway through the model, where this paper had localized reward-model features.
Limitations and open questions
- Selection and scope. About a quarter of prompts gave satisfying insight, and the results are "only claims about specific examples" (Limitations).
- Where the method fails (Limitations, "When Do Our Methods Not Work?"): reasoning without one "crux" token, prompts longer than "about a hundred tokens", long internal reasoning chains, obscure or obfuscated inputs where error nodes dominate, questions of why a model does not do something, and copied words.
- Attention. Attention patterns are taken as given, which hides the crux of the antonym, jailbreak and hint cases; later work addresses part of this.
- Error nodes. In the paper's words, "even the graphs we have highlighted contain significant contributions from error nodes"; their share per graph is not reported.
- Manual and strong interventions. Supernodes are "labor-intensive, subjective" to form, and several effects needed strong or "unnatural" interventions.
- Verification. The model and CLTs are internal, so these graphs cannot be rerun outside Anthropic; open-model work tests the method on other models.
The paper ends with open questions per case (Open Questions), such as how many sequential steps fit in one forward pass and whether its methods can tell faithful from unfaithful reasoning "in the wild".
Research questions
- How common are newline planning sites in models of Haiku's scale? The poem result rests on one couplet, a 25-poem injection test and features found in about half of poems. Running the steering-location sweep on many poems in a comparably sized model, with strengths reported, would show how general it is.
- What share of each case's attribution passes through error nodes? Reporting that share for every published graph, and checking whether failed interventions such as the unknown-name case have high shares, would show how far conclusions depend on the replacement model.
- Do intervention results hold at matched strengths? Strengths range from −2× to 10× or more. Repeating the main interventions at fixed strengths, with random-direction controls, would separate specific effects from generic disruption.
- Is a feature active on every dialogue a sign of trained goals in general? Measuring the same statistic for features tied to intended trained behaviors in a production model would show whether "baked in" features are specific to the planted-goal setup.
- Do the medical findings hold on ambiguous cases? Testing whether inhibiting leading-diagnosis features shifts follow-up questions on cases where clinicians disagree would show whether the internal differential is robust.
- Does the refusal structure differ between model generations? Repeating the global-weight analysis on later models trained with other safety methods would show whether harm-to-refusal wiring depends on the training method.
How it connects
- Circuit Tracing is the companion methods paper; Tracing the thoughts of a large language model covers the summary post for both.
- The paper builds on Towards Monosemanticity and Scaling Monosemanticity (summary post: Mapping the Mind of a Large Language Model), and cites Toy Models of Superposition for why neurons are polysemantic.
- The hidden-goal model is from Auditing language models for hidden objectives. The chain-of-thought case cites Measuring Faithfulness in Chain-of-Thought Reasoning and Question Decomposition Improves the Faithfulness of Model-Generated Reasoning; the jailbreak case cites Many-shot Jailbreaking as an attack that primes compliance.
- Later: Tracing Attention Computation Through Feature Interactions fills the attention gap; Open-sourcing circuit tracing tools reruns two of these cases on open models; Emergent Introspective Awareness in Large Language Models cites the entity case as a self-knowledge mechanism separate from recall; A global workspace in language models and Natural Language Autoencoders revisit the planning and hidden-goal examples.
- This note connects the default-refusal and hidden-goal findings to The persona selection model, which argues that post-training shapes an Assistant persona from pretraining representations; the paper does not cite it.
References
- Lindsey, J., Gurnee, W., Ameisen, E., Chen, B., Pearce, A., Turner, N. L., Citro, C., Abrahams, D., Carter, S., Hosmer, B., Marcus, J., Sklar, M., Templeton, A., Bricken, T., McDougall, C., Cunningham, H., Henighan, T., Jermyn, A., Jones, A., Persic, A., Qi, Z., Thompson, T. B., Zimmerman, S., Rivoire, K., Conerly, T., Olah, C., and Batson, J. (2025, March 27). On the Biology of a Large Language Model. Transformer Circuits Thread. https://transformer-circuits.pub/2025/attribution-graphs/biology.html record · 5 pages
- Ameisen, E., Lindsey, J., Pearce, A., Gurnee, W., Turner, N. L., Chen, B., Citro, C., et al. (2025, March 27). Circuit Tracing: Revealing Computational Graphs in Language Models. Transformer Circuits Thread. https://transformer-circuits.pub/2025/attribution-graphs/methods.html record · 15 pages
- Chen, B., and Lindsey, J. (2025). Life of an unsuccessful jailbreak. In Circuits Updates – April 2025. Transformer Circuits Thread. https://transformer-circuits.pub/2025/april-update/index.html record
- Kamath, H., Ameisen, E., et al. (2025, July 31). Tracing Attention Computation Through Feature Interactions. Transformer Circuits Thread. https://transformer-circuits.pub/2025/attention-qk/index.html record · 4 pages
- Jermyn, A., and Gurnee, W.; edited by Batson, J. (2025). Features & In-context learning. In Circuits Updates – September 2025. Transformer Circuits Thread. https://transformer-circuits.pub/2025/september-update/index.html record · 2 pages
- Hanna, M., and Ameisen, E. (2026). Latent Planning Emerges with Scale. ICLR 2026. arXiv:2604.12493. record
- Jacopin, É. (2026). Planning or Improvisation? Stress-Testing the Poetry Planning Site on Open Models and Open Cross-Layer Transcoders. arXiv:2609.18440v1. record
- Marks, S., Treutlein, J., Bricken, T., et al. (2025). Auditing language models for hidden objectives. arXiv:2503.10965. (Cited as the paper describes it.) record · 12 pages