Circuits Updates – October 2026

A one-item Anthropic interpretability update from the Anthropic Fellows Program, pointing to a LessWrong post that introduces the J++ lens, a revision of the Jacobian lens that fits separate Jacobians to clusters of activations and down-weights the noisy ones. On five word-readout tasks in six open-weight models from 9 to 284 billion parameters, it surfaced intermediate variables more often than the Jacobian lens (55.2% against 35.7% recall@10 in Qwen3.6-27B), and the post uses it to propose a mechanism by which models suppress a concept they are told not to think about.

Kola Ayonrinde (Anthropic Fellows Program) and Jack Lindsey (Anthropic) · 2026-10-29 · original

The October 2026 Circuits Update holds a single item, "Anthropic Fellows Program: The J++ Lens", by Kola Ayonrinde and Jack Lindsey. It revises the Jacobian lens (J-lens), the method introduced in July 2026 in the paper summarized in A global workspace in language models for reading which words an internal activation is disposed to make a model say. The J-lens averages a model's Jacobians over many token positions with equal weight; the item reports that some positions contribute noisy gradients and that fitting separate Jacobians to clusters of activations, then weighting the clusters, gives a lens that "more reliably surfaces intermediate latent variables used for the model’s reasoning" than the J-lens or a later variant, the R-lens. The item is a summary of about 230 words; the results are in a longer post on LessWrong, and all experiments use open-weight models, not Claude.

About this update and its version of record

Circuits Updates are informal posts on Anthropic's Transformer Circuits site. This one asks readers to treat its results "like those of a colleague sharing some thoughts or preliminary experiments for a few minutes at a lab meeting, rather than a mature paper" (introduction). The item's "full post" link points to Ayonrinde and Lindsey, "J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses", posted on LessWrong on 8 October 2026, which bylines Ayonrinde with a personal address and Lindsey with Anthropic and acknowledges Anthropic for compute. That post is the version of record for every number here and is cited by its section headings, figure numbers and table numbers; it is not peer reviewed. The release also includes code (github.com/safety-research/jpp_lens, created 8 October 2026, Apache 2.0 license), the fitted lenses on Hugging Face (koayon/jpp-lenses) and an interactive demo on Neuronpedia for Qwen3.6-27B; none of these was inspected for this note. The post's figures are static images: numbers here come from its text, captions and tables, except where a figure is named as the only source.

Date. The page prints no date. The site's index lists the update under "October 2026", and the index's link to it carries the date 29 October 2026 in its markup (a data-date attribute, read on 13 November 2026); that is the date given here. Archived copies of the index from 27 and 28 October 2026 do not list the update, which agrees with it. The LessWrong post it summarizes appeared earlier, on 8 October.

Terms used in this note

  • Jacobian lens (J-lens). For each layer, the Jacobian of the final-layer residual stream with respect to that layer's residual stream, averaged over source positions, later positions and many texts, then multiplied by the model's unembedding. Applied to an activation, it gives a ranked list of vocabulary tokens the activation is disposed to make the model say. Each token's row is a lens vector. (Definition from the workspace paper as summarized in A global workspace in language models.)
  • Workspace lens. The post's term for any method that, like the J-lens, aims to read only the "workspace": the small set of verbalizable representations that the workspace paper reports a model uses for report and flexible reasoning (post, "Background").
  • R-lens. A J-lens computed with a layer-wise relevance propagation (LRP) backward pass, which adds stop-gradients (for example through normalization layers) to limit error accumulation, introduced on LessWrong on 5 August 2026 by Camila Blank, Agam Bhatia and Neel Nanda.
  • Faithful. The post calls a lens faithful "to the extent that its readouts surface the intermediate variables that the model uses in its reasoning and its lens vectors act as those variables when patched into the model" ("Introduction").
  • Recall@10. The share of test items whose target intermediate word appears among a lens's top ten tokens at any of seven equally spaced layers ("Experimental Setup").

The method

The J++ lens makes three changes to the J-lens ("Methods"):

  1. Jacobian filtering. At each layer, the activations used for fitting are split into eight clusters by k-means, one averaged Jacobian ("Expert Jacobian") is fitted per cluster, and the experts are combined with weights learned on a small development set of latent-variable extraction problems, disjoint from the evaluation set. The post reports that in practice the weights mainly decide which experts get zero or negative weight. It hypothesizes, without testing, that the noisy experts come from regions where the model's layer-to-output map is strongly curved, so a linear approximation is poor.
  2. LRP backward pass, as in the R-lens.
  3. Readout filtering. Tokens with no letter or digit (6,116 of them, such as punctuation and whitespace) are removed from the readout after the fact (Appendix I, Table 5).

The result is still one linear map per layer, so the update calls it "a drop-in replacement for the J-lens". Lenses were fitted on 64 sequences of 128 tokens from the WikiText-103 corpus, skipping the first 16 positions to avoid attention sinks; fitting costs about 7% more compute than a J-lens on the same data, plus a clustering step, and nothing extra at inference (Appendix I).

Models and tasks. The main model is Qwen3.6-27B; readouts are also evaluated on Qwen3.5-9B, Gemma 4 31B, Olmo 3 32B, Qwen3.5-122B-A10B and DeepSeek-V4-Flash (284 billion parameters) ("Experimental Setup"). The readout tasks are five adapted from the workspace paper: multihop, multilingual, typo, association and poetry. A causal "intermediate swap" task, also from that paper, patches in a different concept's lens vector and asks whether the model's answer changes accordingly.

What the post finds

Readout in Qwen3.6-27B. Macro-averaged over the five tasks, recall@10 was 55.2% for the J++ lens against 35.7% for the J-lens and 37.7% for the R-lens (Figure 1; Table 3b). By task, the J++ lens scored 67.3% on multihop, 74.6% on multilingual, 91.7% on typo, 37.3% on association and 4.9% on poetry; the J-lens scored 56.1%, 47.4%, 60.4%, 10.8% and 3.7% (Table 3b). The post notes that poetry "remains hard for every lens" (Figure 2 caption) and that the largest gains are in early layers. On Qwen3.6-27B, the authors fitted the J-lens and R-lens themselves on the same data and budget as the J++ lens, having found the released J-lens for that model weaker than their own (Appendix I, "External Model Usage").

Across six models. The J++ lens scored highest on every model, from 48.4% (Qwen3.5-122B-A10B) to 61.4% (DeepSeek-V4-Flash), against 24.4% to 35.7% for the J-lens; its median relative improvement was 63.5% over the J-lens and 42.2% over the R-lens (Table 1, where all lenses are scored with readout filtering). The relative gain was largest on the two largest models (+98% and +101%), from which the authors "suggest that the advantage of the J++ Lens may grow with scale" ("The J++ Lens Outperforms..."); that is two models. For the other five models the J-lens and R-lens baselines were released lenses fitted by others (Neuronpedia and Camila Blank; the R-lenses with 25 prompts), or, for Gemma and Olmo, R-lenses fitted by the authors, so those comparisons are not all on matched fitting data (Appendix I).

Causal swaps. In 67 intermediate-swap trials on Qwen3.6-27B, patching in a J++ lens vector made the swapped-in concept the model's top answer word in 59.7% of trials, against 49.3% for the J-lens and 50.7% for the R-lens (Figure 2c). The post states the result as J++ vectors being "at least as causally important" as the others.

Which change matters. Removing each change from the J++ lens in turn lowered recall@10 by 2.9 points (readout filtering), 6.6 points (LRP) and 11.7 points (Jacobian filtering); the last is 60% of the 19.5-point gain over the J-lens (Table 2). Of the 52 Expert Jacobians with their own fit, 24 beat a single pooled Jacobian when applied to all items, and 17 scored below 2% recall@10 at their layer ("Why Does the J++ Lens Work?", Figure 3).

Overfitting checks. Because the expert weights are learned on labeled problems, the post reports variants (Appendix H, Table 4): learning the weights with no items from the task being evaluated gave 54.4% (0.8 points below the full lens), and choosing the single best expert per layer gave 50.2%. The authors read these as evidence that the method is "learning a generalisable extraction of the workspace rather than overfitting" to the evaluation set, noting it can only rebalance eight Jacobians per layer. Using more experts per layer raised recall.

Variance. The post reports no repeated fits, seeds or confidence intervals for the readout results. For one case-study comparison it states that the J-lens effect is not statistically significant (Figure 7c), without naming the test.

A case study: retrieve-then-suppress

The workspace paper had reported that a concept the model is told not to think about still often appears in its workspace. Using that paper's directed-modulation task, the post proposes a "retrieve-then-suppress" mechanism: in early layers an instruction about a concept brings it into the workspace whether the instruction is to focus on it or to avoid it, and later layers raise or lower its salience using that early representation ("Case Study"; Appendix A).

  • Readouts by condition. With the J++ lens, instructed concepts rose above a no-instruction baseline by layer 24 under focus, neutral-mention and suppress instructions; the focus condition kept rising to 26% by layer 56, the suppress condition stayed at 7 to 8%, and the neutral condition fell from 14% to 8% between layers 24 and 48 (Figure 5). With the J-lens the concept became visible only from layer 40.
  • Clamping. Removing the concept's lens vector at layers 16 to 28 under the suppress prompt made the concept's rank rise over the next 16 layers instead of fall (Figure 6).
  • How often. Across 440 combinations of concept and carrier sentence, the pattern of a concept entering the top ten readouts early and dropping out later appeared in 11.1% of items (49) with the J++ lens, against 1.8% (8) with the J-lens and 3.4% (15) with the R-lens (Figure 8, printed labels; the share appears only in that figure). The pattern was first noticed with the J++ lens, and the clamping results above are on the items that show it (Figure 9).
  • Naming versus avoiding. In a replication of a workspace-paper experiment, ablating an implied concept's J++ vector in early layers (layers 24 to 35) raised the probability that the model named the concept when asked to avoid it by 2.8 percentage points beyond a control ablation, against 0.36 points for the J-lens vector, which the post says is not statistically significant (Figure 7c).

The post calls this "preliminary evidence" and says the predictions held "in some (but not all) cases of suppression" (Appendix A). It also argues that the mechanism would have been hard to see with the J-lens.

What the post says about the workspace hypothesis

The authors list four explanations for why the J-lens misses intermediate variables one would expect in the workspace, from the workspace being a poor model of the network to the lens being an imperfect tool, and read their gains as evidence for the last: "a substantial share of these failures may come from the J-Lens being an imperfect tool" ("Discussion"). They also note, in an appendix on what the work means for questions of AI consciousness, that if many differently built lenses all show workspace-like properties, workspaces may "come on the cheap" and be less telling; they say they are uncertain about the implications (Appendix C).

Limitations and open questions

  • Single tokens only. The lens reads single vocabulary tokens; arithmetic could not be evaluated on Qwen models because they tokenize numbers digit by digit ("Limitations").
  • Unknown ceiling. A model may "shortcut" to an answer without representing the intermediate variable, so the best achievable score is unclear ("Limitations").
  • Monitoring not shown. A footnote states that it is "as of yet unclear how effective the Workspace Lens is for monitoring" (footnote 11), though the post presents evaluation awareness and unverbalized scheming as motivating uses.
  • Task-specific weights. Asked on LessWrong whether weights learned on single-word extraction problems generalize to open-ended audits, Ayonrinde replied (6 November 2026) that on the WorkspaceBench evaluation set the J++ lens scored 0.9 against 0.73 on an agentic-misalignment scenario and 7.2 against 4.5 (J-lens) and 5.2 (R-lens) on multi-token tasks, and wrote "I don’t think we see evidence of overfitting on particular extraction tasks". These numbers appear only in the comment; WorkspaceBench was not read for this note.
  • Open models, one main model. Ablations and the case study use Qwen3.6-27B only; no Claude model is tested.

Research questions

  1. Does the J++ lens's gain hold on Claude models and on the workspace paper's own tasks? The J-lens was introduced on Claude Sonnet 4.5; the J++ lens is tested only on open-weight models. Repeating the readout and swap evaluations on the original models and prompts would show whether the gain transfers.
  2. Is the scale trend real? The suggestion that the advantage grows with scale rests on two models whose baselines were fitted by others. Fitting all three lenses on matched data across a series of sizes from one model family would test it.
  3. What makes an Expert Jacobian noisy? The post hypothesizes curvature and finds some experts dominated by punctuation positions, but no clear pattern. In a LessWrong reply (6 November 2026), Ayonrinde reported that rotating the experts into alignment with the best one before averaging recovered about 30% of the J++ lens's gain, and judged curvature or linearization error the likelier cause. Measuring linearization error per cluster and correlating it with each expert's recall would test the curvature account.
  4. How variable are the results? No repeated fits are reported. Refitting the clusters and weights with several random seeds and reporting the spread of recall@10 would show how stable the 55.2% is.
  5. Does retrieve-then-suppress hold across concepts and models? The readout pattern appears in about 11% of items with the J++ lens, on one model. Reporting the share of items that show each predicted pattern, with intervals, across several models would show how general it is.
  6. Does a better lens improve monitoring? The post's motivation is detecting evaluation awareness and scheming. Comparing J-lens and J++ lens detection rates on transcripts with known labels would answer whether readout gains carry over.

How it connects

  • Within this library. The J++ lens revises the method of the paper summarized in A global workspace in language models, adopting its tasks and swap protocol; that note records Neel Nanda's invited commentary, which replicated parts of the paper on Qwen 3.6 27B, the model the J++ post uses for most of its experiments. The post compares workspace lenses with sparse autoencoders (Towards Monosemanticity, Scaling Monosemanticity) and with natural language autoencoders, which it describes as more expressive but more prone to confabulation and more expensive (Appendix B).
  • Within this batch of updates. Circuits Updates – June 2026 is also an Anthropic Fellows Program item, on turn-averaged sparse autoencoders; the two items do not cite each other.

References