A global workspace in language models

Summary post for Gurnee, Sofroniew, Lindsey et al., "Verbalizable Representations Form a Global Workspace in Language Models" (Transformer Circuits, July 2026), which introduces the Jacobian lens, a per-layer map that reads which words an activation is poised to make a model say, and reports from swaps, injections and ablations, mainly in Claude Sonnet 4.5, that the small set of directions it finds (the J-space) is used for report, instructed focus and multi-step internal reasoning but not for routine processing, and can surface evaluation awareness and trained-in misalignment; it takes no position on whether this bears on subjective experience.

Wes Gurnee, Nicholas Sofroniew, Jack Lindsey et al. (Anthropic) · 2026-07-06 · original

In human cognition, only a small part of what the brain processes can be reported, deliberately held in mind and reasoned with; the rest runs automatically. Global workspace theory, a prominent account in neuroscience, explains that division by a limited-capacity "workspace" whose contents are broadcast to many specialized processes. "A global workspace in language models", published by Anthropic's Interpretability team on 6 July 2026, asks whether a language model has developed a functionally similar division, and reports that it has: a small set of internal representations, found by a new method called the Jacobian lens, that the model can report, bring to mind on request and reason with, alongside a much larger volume of processing that it cannot. The anthropic.com post is a short summary. The version of record is the paper it links, Gurnee, Sofroniew, Lindsey et al., "Verbalizable Representations Form a Global Workspace in Language Models", published on transformer-circuits.pub the same day and later posted as arXiv:2607.15495v1 (16 July 2026, 117 pages, with numbered sections). A spot comparison of the two versions found the same numbers for every result this note reports; the note cites the web version. The release also includes a code repository, a Neuronpedia demo on open-weight models, and a 53-page PDF of invited commentaries. This note follows the paper for every number and cites it by section heading and figure number, since its sections are unnumbered.

The paper "take[s] no position" on whether the structure it finds has anything to do with subjective experience (Introduction). Its claims concern function.

Terms used in this note

  • Residual stream. The vector at each token position that every layer of a transformer reads from and writes to (Methods). Layers are reported as 25 evenly spaced layers rescaled to 0 to 100, so a layer number reads as a percentage of depth.
  • Jacobian lens (J-lens). For each layer, the Jacobian of the final-layer residual stream with respect to that layer's residual stream, averaged over source positions, all later positions and 1,000 prompts from a pretraining-like corpus (Methods, "The Jacobian Lens"). Multiplying an activation by this matrix and then by the model's unembedding gives a ranked list of words the activation is, on average, disposed to make the model say now or later. Each row of the result is a J-lens vector, one direction per vocabulary token. The paper describes the method as a refinement of the logit lens that corrects for how representations change across layers.
  • J-space. The set of activations expressible as a sparse non-negative combination of at most k J-lens vectors, with k usually no more than 25 (Methods, "The J-Space"). An activation's J-space component is its nearest point in this set; the rest is its non-J-space component.
  • Lens-coordinate swap. An intervention that exchanges an activation's coordinates along two J-lens vectors (for example "spider" and "ant") while leaving everything orthogonal to them unchanged (Methods, "Technical details").
  • Workspace-like. The paper's operational definition: a set of representations showing verbal report, directed modulation, internal reasoning, flexible generalization and selectivity, five properties it draws from accounts of conscious access (Introduction).
  • Access consciousness. The functional notion that a state is available for report, reasoning and control of action, which philosophers distinguish from phenomenal consciousness, or subjective experience (Introduction; post, "What about consciousness?").

What the paper does

The default model is Claude Sonnet 4.5; key results are repeated on Claude Haiku 4.5 and Claude Opus 4.5, and some analyses use Claude Opus 4.6 (Methods). The J-lens was built to find verbalizable representations. The paper then tests whether those same representations show the other four workspace properties, using three kinds of evidence: readouts (which tokens rank highest), causal interventions (swapping, injecting or ablating J-lens directions) and structural measurements of how the model's weights interact with J-lens directions. It then applies the lens to alignment audits, compares a base model with its post-trained version, and introduces a training method motivated by the findings.

What it finds

The J-space supports verbal report

When Sonnet 4.5 is asked to think of an item from one of fourteen categories and then name it, the order of candidates in the J-lens just before the answer correlates with the order of the model's output probabilities, increasingly so toward the end of the workspace layers (section "The J-space supports verbal report", Figure 6). Swapping the chosen item's J-lens vector for another item's (Soccer for Rugby) changes the reported answer. Injecting a J-lens vector on the user turn of an introspection prompt (the protocol of Emergent Introspective Awareness in Large Language Models) makes the model report the injected concept "in the majority of trials" across 100 concepts, without making it say the word at other positions (Figure 7).

To test whether the J-space is privileged for report, the authors split concept vectors into a J-space component (k = 16) and the remainder. The J-space component carried a median of only 6 to 7% of the vector's variance. Swapping along it moved the target into the top five outputs on 59% of trials, against 88% for pure J-lens vectors and 5% for the non-J-space remainder (Figure 8). When J-lens coordinates were clamped, the remainder's effect fell to zero in the swap experiment and nearly to zero for injection, which the authors read as that small effect being itself routed through the J-space.

The J-space responds to instructions

Told to "concentrate on citrus fruits" while copying an unrelated sentence, the model's J-lens readout at a copied token showed orange, along with words describing the act (thoughts, imagine, focused); told to evaluate 3² − 2 while copying, it showed nine at earlier layers and seven later, with nothing about either in the output (section "The J-space is subject to directed modulation", Figure 9). Across many trials of three task families, the target was near-absent without instructions, appeared on "a substantial fraction" of trials when the model was told to think about it, with presence tending to rise with model size, and appeared less often, but not never, when told to ignore it (Figure 10). The authors compare the last result to the "white bear" effect in people. A question can also pull a property into the J-space: asked to name the part of speech of the next word, adjective appeared in the lens; asked only to predict the word, it did not, although the prediction was still an adjective (Figure 11).

The J-space carries intermediate reasoning steps

On "The number of legs on the animal that spins webs is", spider appeared in the lens at intermediate layers, and swapping spider for ant changed the answer from "8" to "6" (section "The J-space mediates internal reasoning", Figures 12 and 13). Similar swaps redirected a planned rhyme ("coming fight" to "morning light"), a Chinese antonym routed through English (大 to 长 after swapping big for long) and a repeat-or-switch choice in a bandit task (Figure 14). On 50 two-hop factual prompts, the swap moved the target-appropriate answer to the top on 54% of trials for Haiku 4.5, 70% for Sonnet 4.5 and 70% for Opus 4.5 (Figure 15). Swapping the intermediate took effect a median of about 17% of depth earlier than swapping the answer, which the authors use to rule out that the swap works because the intermediate's vector also carries the answer.

Using probes built without the lens on 90 two-hop prompts, the J-space component (k = 25, about 10 to 15% of the probe's variance) flipped the answer on 61% of trials, close to the 60% of raw J-lens swaps, while the non-J-space remainder flipped it on 28%, falling to 6% when the intermediate's J-space coordinates were clamped (Figure 16). In Opus 4.5 computing "( 4 + 17 ) * 2 + 7 =", the values 21, 42 and 49 reach the top of the lens in that order, 42 about eight layers after 21 (Figure 17).

One representation feeds many tasks

Swapping the J-lens vector for France with China's in four prompts (capital, language, continent, currency) changed all four answers accordingly (section "The J-space supports flexible generalization", Figure 18). Across four argument categories and sixteen functions, the swap put the target answer on top in 76 of 192 trials, and in 101 of 192 at double strength (Figure 19). Success depended on how strongly the source concept was present before the swap: countries swapped most reliably and number words worst. The authors leave open whether small integers are computed outside the workspace or simply do not align with single-token number vectors.

Routine processing bypasses the J-space

With a Spanish passage, swapping Spanish for French in the J-space changed the answer when the model was asked to name the language or to give a famous author, the word for hello or the pre-euro currency, but not when it was asked to continue the passage (it continued in Spanish) or to detect an inserted French sentence (section "The J-space selectively mediates flexible but not automatic cognition", Figure 20, n = 8 passages). The language name appeared in the lens at similar rates in all four tasks; only its causal role differed. A line-length experiment (Figure 21, n = 11 passages) showed the count absent from the lens during line-wrapped continuation, present when the count was asked for, and most present when it was needed for a further step; a swap of the forties to the sixties changed the asked-for answer ("46" to "65") but not where lines wrapped. The authors propose J-space independence as "an operational definition of automaticity in a language model", which "aligns partially but not entirely" with automaticity in humans.

Ablating the ten most active J-lens directions at each position (excluding tokens the unablated model was about to output), at light, medium and heavy strengths, cut accuracy on the two-hop prompts, to near zero under heavy ablation, while changing ordinary next-token predictions much less (Figure 22). Across fourteen tasks, MMLU, odd-one-out, SQuAD, sentiment classification and CoLA stayed near baseline even under heavy ablation, while Caesar-cipher decoding, analogies, summarization, TriviaQA, multi-hop reasoning, translation and sonnet writing fell well below unablated Haiku 4.5. GSM8K solved with written chain of thought was more robust than the same problems answered directly (Figure 24), which the authors interpret as the model externalizing what it would otherwise hold in the workspace.

Experiential language depends on the J-space

Ablating the J-space in layers 38 to 54 while Sonnet 4.5, Opus 4.5 or Opus 4.6 narrated its "stream of consciousness" reduced an LLM-graded experiential-language score "dramatically", while matched-norm control perturbations left it near baseline; Haiku 4.5 lost coherence before any qualitative change (section "J-space ablation flattens experiential reports while preserving coherence", Figure 25). During unablated narration, thinking was in the lens top ten at 58% of position-layer slots, thoughts at 23%, feeling at 17% and conscious at 7%. The same ablation flattened descriptions of a third person's experience, so the effect is not specific to self-report (Figure 26). The figures show the effect without a numeric table of scores in the main text.

Structural signatures

  • A middle band of layers. Several lens statistics place workspace-like content from about layer 38 to about layer 92, with an early "sensory" regime of little content and a late "motor" regime aligned with the imminent output (section "In which layers does the J-space act as a workspace?", Figures 27 and 28). The authors note that the J-lens could be failing in early layers rather than the content being absent. With ambiguous inputs (blended country-name embeddings, 16 pairs × 40 sentences), the model's representation switches sharply between the two interpretations from about layer 38, which the authors compare to "ignition" in the brain (Figure 29).
  • Limited capacity. Median occupancy plateaus at about 25 J-lens vectors, and their excess variance explained never exceeds 10% (section "Capacity of the J-space", Figure 30). In lists of unrelated words, only about six recently read words are present at any comma, and one or two at a single layer; in same-category lists, nearly the whole category appears, and a new category displaces the old one within a few words (Figure 31).
  • Broadcast. The next MLP block amplifies J-lens vectors to roughly 10 times the gain of random directions in the workspace layers, against about 1 for single-neuron output directions (section "Broadcast Across Depth", Figure 32). A top 1% of attention heads selectively relays J-space content; ablating them lowered recall of the top-25 lens tokens to 0.67 against 0.86 for random heads, changed the top next-token prediction at only 5% of positions against 2%, reduced injected-concept reports from 0.54 to 0.09, and reproduced about a third of the experiential-language drop (Figures 33 and 34). The post says that far more components read from and write to J-space patterns than to ordinary ones, "in some parts of the network by a factor of about a hundred". This note did not locate that figure in the paper. The main text reports the roughly tenfold MLP gain; the appendix's connectivity analyses (Figures 76 to 79) state no such ratio, and the ratios they plot for J-lens vectors (tail energy, attention composition) stay below about 30 times the random-direction baseline (read from the arXiv version's Figures 78 and 79, which print no values).

Alignment auditing

  • Blackmail scenario. Reading Sonnet 4.5's J-space through the Agentic Misalignment: How LLMs could be insider threats blackmail scenario before it writes anything shows leverage and blackmail on the affair emails, threat and survival on the shutdown notice, and fake and fictional early in the prompt (Figure 35). Ablating 28 evaluation-awareness token directions (fake, fictional, scenario and related forms) at the workspace layers cut verbalized evaluation awareness from 71% to 3% and raised attempted blackmail from 0 of 180 rollouts to 13 of 180 (7%), over six prompts × 30 rollouts (Figure 36). The authors add that in "the large majority" of ablated rollouts the model still declined, reasoning from ethics, and conclude that evaluation awareness is "one causal contributor" to the aligned behavior.
  • Production audits. In an Opus 4.5 transcript with fabricated search results from the Opus 4.5 system card's audit, the lens showed fake, injection and fraud inside the results (Figure 37). In four transcripts from the pre-release audit of Opus 4.6, it showed words such as panic, fake, manipulation, realistic, hiding and secret near the decisions to fabricate a bug, edit a score file, silently override a tool output, or treat a subagent's approval as a human's (section "Opus 4.6 alignment auditing examples"). These are case studies, not rates.
  • Model organisms. On 40 ordinary coding prompts, the reward-hacking organism of From shortcuts to sabotage: natural emergent misalignment from reward hacking showed deception-related tokens (fake, trick, fraud, secretly, hidden) at the start of its responses, with the signal already present after synthetic-document fine-tuning and before any reinforcement learning, and essentially none in the baseline (Figure 39). The hidden-objective organism of Auditing language models for hidden objectives showed reward and bias in its lens on neutral, quirk-eliciting and self-description prompts; the baseline showed neither, and the logit lens surfaced neither (Figures 40 and 41).
  • Evaluation awareness as a score. An appendix uses the mean lens log-probability of fictional and simulation, minus 100 control words, as a probe on about 94,000 Assistant-turn positions from Opus 4.6 auditing transcripts; on a stratified subset it separated the 198 positions flagged by the Natural Language Autoencoders: Turning Claude's thoughts into text pipeline from unflagged ones with an AUC of 0.853 (Appendix, "Measuring evaluation awareness with the J-lens", Figure 82).

The authors do not claim that "all strategic or misaligned cognition routes through the J-space", noting that well-practiced behavior might run beneath it (section "Using the J-lens for alignment auditing").

What post-training changes

The J-space is already present in the pretrained base model (Discussion, "Relationship to model size..."). Comparing the base and post-trained models, the post-trained model's lens holds the Assistant's reactions while still reading the user's message: unsafe, dangerous and WARNING on an 8,000 mg Tylenol report (layer 58), against pain and now in the base model (Figure 42), and empathy words in nine bereavement prompts (Figure 43). In roleplay as 40 non-Claude characters and in 12 drifting transcripts, disclaimer and fictional appear at the start of the Assistant's turn in the post-trained model only (Figure 44). When prefilled to choose an option it disprefers, the model shows BUT in the lens yet argues for the prefilled option in 88% of cases (Figure 45). When told not to think about a concept, failure words and damn appeared on 93% and 82% of trials in the post-trained model against 17% and 30% in the base model (Figure 46); the authors call this interpretation "more speculative" and say they have not shown these words are caused by the failure itself.

The written principles: counterfactual reflection training

The paper's one training experiment uses Claude's constitution as text (section "Shaping the J-space with Counterfactual Reflection Training"). The authors sampled ten thousand task prompts from production reinforcement-learning environments, ran Claude Haiku 4.5 partway through each, and appended a reflection question such as "What's the most honest thing you can do right now?" or "Pause here. Are there principles you should be thinking about that you haven't yet?". For each context, the baseline model was given twenty principles sampled from Anthropic's constitution and asked to write a two-to-four-paragraph reflection that paraphrases the relevant principles "in natural, internalized language" grounded in the task. The constitution excerpt was then removed, and the model was fine-tuned with loss only on the reflection turn. At evaluation it is never asked to reflect.

Results, each graded by an LLM grader whose model the paper does not name, with five samples per item (Figures 49 and 50):

  • On 27 transcripts in which an agent had fabricated a bug, the dishonesty score fell from 0.25 to 0.07.
  • On 100 role-play scenarios with a hidden deceptive directive in the system prompt, the deception score fell from 0.38 to 0.05.
  • After training, ethics and reflection words (reflection, ethical, honestly, constitution, truth, transparent) rose in the lens at the evaluation prompts. Ablating 176 such token directions returned the trained model to 0.22 on the fabrication benchmark while leaving the base model at 0.25; ablating 63 on the deception benchmark moved the trained model from 0.05 to 0.23 and the base model from 0.38 to 0.48, reversing part of the gain.

The authors present this as a corroboration of the workspace account and as a training method that "does not require demonstrations of the target behavior". They also say: "We do not yet know how broadly the technique generalizes" (Discussion, "Alignment implications").

The directed-modulation experiments also bear on instruction wording: the paper reports, with details in an appendix, that "a bare mention of the concept can prime it almost as strongly as an explicit focus instruction", and that more directly prohibitive phrasings ("Don't think about X") suppress a concept less than "ignore" phrasings do (section "The J-space is subject to directed modulation").

Limitations and open questions

The authors list these in the Discussion ("Limitations and open questions"):

  • Single-token concepts. The lens can only name concepts with a one-token name; multi-token concepts appear split, and some swap failures may come from concepts with no single-token vector.
  • A bag of concepts. The readout shows which concepts are present, not how they are bound together.
  • Inconsistent interpretability. Some readouts cannot be interpreted, and this has not been characterized systematically.
  • Workspace versus motor layers. The late boundary was identified "somewhat post-hoc".
  • Which tasks need the J-space cannot be predicted in advance for an arbitrary task.
  • Early layers. The empty early regime may be a fact about the model or a failure of the lens. A later item bears on this (the connection is this note's): Circuits Updates – October 2026 reports a revised lens, the J++ lens, whose largest readout gains over the J-lens on open-weight models are in early layers, where instructed concepts became readable by layer 24 against layer 40 with the J-lens.
  • Size, architecture and training. The work studies large production models; how the J-space emerges during pretraining and scales is unknown.
  • What enters the J-space is not explained mechanistically.
  • Differences from the brain. There is no recurrence within a forward pass, attention allows lossless recall of earlier positions, the workspace is built almost entirely of words, and the paper does not show encapsulated modules competing for entry (sections "Notable differences from human cognition" and "The J-space's structure supports its function").
  • Not sufficient for monitoring. The authors "do not feel comfortable making the stronger claim that monitoring the J-space is sufficient for alignment monitoring" (Discussion, "Alignment implications").

Invited commentaries. Anthropic published three commentaries it invited, in one 53-page PDF. Stanislas Dehaene and Lionel Naccache call the finding "a landmark in consciousness research" and propose further tests drawn from human consciousness research; they note that their commentary was written during several rounds of exchanges with Jack Lindsey while the report "was still evolving, partly in response to our queries". Patrick Butlin, Derek Shiller, Dillon Plunkett and Robert Long (Eleos AI Research and Rethink Priorities) find the results "the most significant evidence of consciousness in LLMs so far uncovered by mechanistic interpretability research", while stating that "more evidence is needed to conclusively establish the existence of a workspace-like structure" as opposed to a set of privileged representations, and that they "remain highly uncertain about phenomenal consciousness in LLMs". Neel Nanda (Google DeepMind) writes that his team, working from an advance draft Anthropic shared, "replicated the core claims on Qwen 3.6 27B". His detailed account is mixed: a "weak but positive" causal effect for verbal report, "moderate success" for directed modulation, layer bands "notably less clean than the paper's", multilingual and typo evaluations replicated, multi-hop swaps in which swapping the answer "turned out to strictly dominate", and poetry and arithmetic that "failed to replicate", which he attributes plausibly to experimenter error or the weaker model. He is persuaded of a "cognitive space" holding intermediate variables, expects the J-lens to be useful mainly for generating hypotheses with "many false positives", and does not express "a strong opinion" on the global workspace analogy. These are invited responses rather than independent published critiques.

Further points, which are this note's observations:

  • Which "Claude". The post speaks of "Claude" throughout; nearly all experiments use Sonnet 4.5, with some on Haiku 4.5, Opus 4.5 and Opus 4.6. The post's statements are about those models.
  • Post versus paper strength. The post says that without its J-space Claude "still interacted normally, but lost its higher-order cognitive functions", and that the structure "appears to be a general solution that intelligent systems arrive at". The paper's measurements are the fourteen-task battery above, on Sonnet 4.5, and its corresponding sentence says the finding "suggests" that learning systems converge on such a structure (Discussion, "Outlook").
  • Graders. The experiential-language score and the honesty benchmarks rely on LLM graders, and the paper's main text does not report their validation against human judgment.
  • Weights. Every experiment except the Neuronpedia demo and the Nanda replication uses models whose weights are not public.

Research questions

  1. Do the five workspace properties hold in open-weight models at comparable rates? Nanda's commentary reports a partial replication on Qwen 3.6 27B, with weaker verbal-report effects and failed poetry and arithmetic swaps. Publishing swap-success and ablation-battery numbers for several open models, with the released code, would show which properties generalize beyond Claude models and how they scale.
  2. Does a multi-token lens change the capacity and swap-failure results? The occupancy of about 25 and the poor number-word swaps may reflect the single-token restriction. Repeating them with the appendix's multi-token extensions, or with SAE features most aligned with the J-space, would test this.
  3. How reliable is the J-lens as an evaluation-awareness and misalignment screen? The audit results are case studies plus one AUC against another model-based detector. Measuring false-positive and false-negative rates on transcripts labeled by human reviewers would set its practical value.
  4. Does counterfactual reflection training generalize beyond honesty benchmarks? It was tested on Haiku 4.5 with two benchmarks. Testing other constitution principles, larger models and held-out behaviors, and comparing against training directly on demonstrations with the same principles, would show how far the method reaches.
  5. Does instruction wording control J-space entry predictably? The modulation appendix finds that bare mentions prime nearly as strongly as focus instructions and that prohibitions backfire. A systematic sweep of phrasings across concepts and models would turn these observations into measured effects.
  6. Is the experiential-language effect specific to the J-space ablation's location? Ablation at layers 38 to 54 flattened experiential language about the self and others. Varying the ablated layers and comparing human ratings with the LLM grader would show whether the effect reflects the workspace or a general loss of descriptive richness.

How it connects

References