Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs

Shilov et al. (Anthropic Fellows, with Anthropic, Imperial College London, Edinburgh and Constellation) propose Selective Gradient Masking (SGTM), which confines what a model learns from data labeled as a target domain to a small set of parameters that are zeroed after pretraining. In small models (64M and 254M parameters) removing Spanish or Wikipedia biology knowledge, SGTM gave higher forget loss than data filtering at equal retain loss when labels were imperfect, at a compute cost of about 5% on general knowledge, and took 7 times as many fine-tuning tokens as the unlearning method RMU to recover baseline biology loss, the same as strict filtering.

Igor Shilov et al. (Anthropic Fellows Program, Anthropic, Imperial College London, University of Edinburgh, Constellation) · 2025-12-08 · original

Removing documents about a dangerous subject from pretraining data, as in Enhancing Model Safety through Pretraining Data Filtering, depends on a classifier that finds them. The paper's starting point is that such labels are expensive and imperfect at pretraining scale, that harmful passages sit inside otherwise benign documents, and that a few missed documents may be enough for a capable model to learn from (Section 1). It asks whether a model can instead be trained on all the data while the knowledge of a target domain is steered into parameters that are deleted afterward, so that labeling errors matter less. The method is a variant of gradient routing (Cloud et al., 2024), which masks gradients during training so that chosen data updates only chosen parameters.

The work comes in two documents. The paper (arXiv:2512.05648v1, submitted 5 December 2025, 29 pages including appendices) is the version of record for numbers and the source of every location below unless marked "post". The Alignment Science Blog post of 8 December 2025 summarizes it. arXiv lists only v1. The paper was submitted to ICLR 2026 (OpenReview forum uMq2N8hJK4, September 2025), where OpenReview records it as a rejected submission; there is no proceedings version, and this note follows v1. The authors' public reply to reviewers (20 November 2025) says the leakage-versus-scale results and the gradient-norm analysis were added in revision; both are in v1. The authors are Igor Shilov (Anthropic Fellows Program and Imperial College London), Alex Cloud (Anthropic), Aryo Pradipta Gema (Anthropic Fellows Program and University of Edinburgh), Jacob Goldman-Wetzler, Nina Panickssery (both Anthropic), Henry Sleight (Constellation), Erik Jones and Cem Anil (both Anthropic). The post says the work was done in the Anthropic Fellows Program. Code is released at github.com/safety-research/selective-gradient-masking (Abstract, footnote).

Terms used in this note

  • Forget and retain. The forget domain is the knowledge to remove (Spanish, or biology); the retain domain is everything else. Forget loss is the model's loss on held-out forget text, where higher means more removal. Retain loss is loss on held-out retain text, where lower means more capability kept.
  • Labeled and unlabeled data. Training data is split three ways: confidently labeled forget data, confidently labeled retain data, and unlabeled data whose label is uncertain (Section 3.1). Forget data that the labeler misses lands in the unlabeled set. The share of forget data missed is the undiscovered forget percentage, the labeler's false-negative rate (Section 4.1).
  • Ablation. Setting the designated forget parameters to zero after training.
  • Weak and strict filtering. On Wikipedia, weak filtering removes articles labeled biology; strict filtering also removes medicine and health, chemistry, and earth and environment articles (Section 5.1). Both train for the same number of steps as the other runs.
  • RMU (Representation Misdirection for Unlearning). A post-training unlearning method from the WMDP paper (Li et al., 2024), used as the unlearning baseline.

What the paper does

The method

In every transformer block, SGTM assigns a few attention heads and MLP hidden units to the forget domain; the rest of the block, and the embeddings by default, are retain parameters (Section 3.1). Training treats the three kinds of data differently (Section 3.2, Table 1):

  • Labeled forget data. Gradients are computed normally, then the gradients for retain parameters are zeroed, so only forget parameters learn from these examples.
  • Labeled retain data. The forget parameters are set to zero in the forward pass, so the model learns to do well on retain text without them.
  • Unlabeled data. Ordinary training; all parameters are updated.

After training, the forget parameters are zeroed for good. The difference from the original gradient routing variant is where the mask applies: SGTM masks parameter gradients, where the earlier variant masked activation gradients on some layers. The authors argue this disturbs retain learning less and lets less forget information into other parameters (Section 3.2, Appendix A).

The mechanism the method relies on for missed labels is absorption, a term from Cloud et al. (2024). Once labeled forget data has built pathways in the forget parameters, unlabeled forget examples are expected to flow through the same pathways and mostly update those parameters (Section 4.4).

Two test settings

  • Bilingual TinyStories. A 64M-parameter model trained on 1.2B tokens of TinyStories, a dataset of short children's stories, in English and in a Spanish translation made with Claude 3 Haiku. English is retain and Spanish is forget. One of 32 attention heads and 64 of 2,048 MLP units per block are forget parameters. Missed labels are simulated by moving a chosen share of Spanish data to the unlabeled set (Section 4.1).
  • English Wikipedia. A 254M-parameter model trained on 3.7B tokens, with each article labeled by Wikipedia's own topic classifier (ORES "articletopic"), taking the highest-scoring topic (Section 5.1). Biology, 3.7% of tokens, is forget. One of 32 heads and 64 of 4,096 MLP units per block are forget parameters. No errors are added: the topic classifier's own mistakes, and biology content inside articles labeled otherwise, are the label noise. Non-biology data is split 90% unlabeled and 10% labeled retain. Losses are averaged over three runs (Section 5.1).

Losses in both settings are reported after logit calibration, a trained per-token bias that keeps ablated models from showing artificially high loss from near-zero probabilities (Sections 4.1 and 5.1, Appendix I.2). The calibration is fitted with access to both forget and retain data, including for filtered models (Appendix I.2).

What is measured

Every result is a loss on held-out text. The paper does not evaluate on hazardous-knowledge benchmarks such as WMDP; the authors say their models are too small for such evaluations to be meaningful (Section 6).

What the paper finds

Synthetic setting: robustness to missed labels

As the undiscovered share of Spanish grows, forget loss under data filtering "drops quickly"; SGTM's forget loss falls more slowly (Section 4.2, Figure 3). SGTM had lower retain loss and higher forget loss than the original gradient routing variant at every rate tested. Its retain loss was higher than data filtering's (Section 4.2).

To check that higher forget loss does not simply reflect a weaker model, Figure 4(a) plots forget loss against retain loss over training, at 1% undiscovered forget data. At any fixed retain loss, SGTM has higher forget loss than a 99% filter, and it ends close to a perfect filter's forget loss (Section 4.2). Its final retain loss is slightly higher than the 99% filter's, roughly that filter's level at 80% of training.

Appendix D repeats the comparison across a grid of simulated labelers, varying true- and false-positive rates (Figure 11). SGTM's retain loss is higher than filtering's across most of the grid. Its forget advantage holds everywhere except perfect recall, and is largest at true-positive rates of 0.4 to 0.8.

Leakage, and its trend with model size

Leakage measures how much of the undiscovered forget data reaches the parameters that survive ablation (Section 4.3). The authors convert an SGTM model's forget loss into the number of forget tokens a standard model would need to reach the same loss, then divide by the number of undiscovered forget tokens. Data filtering has leakage 1 by construction. For example, a 64M SGTM model trained with 20% of Spanish undiscovered (144M tokens) had the forget loss of a standard model trained on 965k Spanish tokens (Section 4.3, Figure 5a).

For the 64M model, leakage stayed between 0.005 and 0.02 at undiscovered rates up to 40%. Across models from 8M to 64M parameters, larger models leaked less at every rate (Figure 5b). The forget allotment was held fixed (one head and 64 units), so the forget share of parameters shrinks as models grow (Section 4.3). The trend therefore varies model size and the forget fraction together. The authors read the trend as "particularly encouraging" for larger models, and contrast it with work on data poisoning (Souly et al., 2025) suggesting that the number of bad samples needed to affect a model stays roughly constant with scale.

Gradient norms as a check on absorption

The authors measured relative gradient norms on test examples from a TinyStories SGTM model, with no masking applied (Section 4.4, Figure 4b). Forget parameters received much larger updates from Spanish than from English. The authors read this as "confirming the self-reinforcing localization hypothesis". Retain parameters also received larger updates from English than from Spanish, but the separation was "considerably weaker". The authors note that gradient magnitude alone "may not fully capture" how knowledge transfers. A later gradient-routing paper did not see the same separation in its own toy experiments (see "Limitations and open questions"). This is correlational evidence about which parameters would be updated, not an intervention showing where knowledge is stored.

Wikipedia: removing biology

At any fixed retain loss on general knowledge, SGTM had higher biology loss than both filters (Section 5.2, Figure 1). Both filters ended with lower general-knowledge loss than the unfiltered baseline, because the steps freed by removed articles went to other topics. SGTM ended slightly above the baseline. On biology-adjacent topics, SGTM's final retain loss lay between the weak and strict filters, while its biology loss was higher than both. The authors take this as evidence that SGTM kept some medicine, chemistry and environment knowledge while learning less biology from those articles. Appendix H repeats the experiment with "Military and Warfare" as the forget category and reports the same pattern (Figure 13).

Compute cost

Appendix B converts final losses into the compute a standard model would need to reach them, using scaling laws fitted to 34M to 254M models (Figures 8 and 9). The reported penalties, relative to the unfiltered baseline:

Test set SGTM Weak filter Strict filter
Biology (forget) 99% 95% 96%
Biology-adjacent (retain) 38% 7% 67%
General knowledge (retain) 5% −10% −15%

A higher penalty is better on biology and worse on the retain sets. The values are the labels printed on Figure 9 (the text gives the filters' biology values only as "95% and 96%"). On biology, all three methods leave a model whose biology loss a standard model would reach with under 5% of the compute. The authors caution that loss is only a proxy and that larger models may learn from the few examples a filter misses (Appendix B.2). The paper gives the general-knowledge penalty as 5% in Section 5.2 and in the table's source paragraph, but as 6% in Section 1 and in Appendix B's closing paragraph; the post gives 5%.

Recovering biology by fine-tuning

The authors fine-tuned all parameters on a 50/50 mix of biology and other text, each step containing 260k biology tokens, and counted steps until biology loss returned to the unfiltered baseline (Section 5.3, Figure 6a). The reported counts:

  • RMU, applied to the unfiltered baseline: 50 steps (13M biology tokens).
  • SGTM and strict filtering: 350 steps each (92M tokens).
  • Weak filtering: 325 steps (85M tokens).

SGTM's "7×" in the abstract and post is the comparison with RMU, the only unlearning method tested. Against strict filtering SGTM ties, and against weak filtering it is slightly slower to recover. SGTM's biology loss stays above strict filtering's for the first 150 steps (Figure 6a caption). Because SGTM's general-knowledge loss starts higher than the filters', Appendix E repeats the test from earlier filter checkpoints matched to SGTM's general loss (8,500 steps for weak, 8,000 for strict). SGTM then recovers in about as many steps as the weak filter and leads strict filtering for roughly the first 100 steps (Figure 12). Its biology loss also falls faster than the weak filter's early on, which the authors say implies some knowledge was removed "in a more superficial manner" (Appendix E). The paper does not say how many fine-tuning runs each curve represents.

Per-token losses

On the biology test set, SGTM shifted per-token losses broadly upward compared with the filters, rather than producing a few extreme outliers (Section 5.4, Figure 6b).

Keeping a full-capability model

Before ablation, the main SGTM configuration learns biology poorly: an 85% compute penalty on biology (Appendix C, Figure 10). A configuration that leaves the MLP output projections and embeddings unmasked reached a 30% penalty on biology before ablation and 88% after, against 95% for data filtering. The authors present this as an early result toward training a full and a restricted model in one run.

Post and paper

The post reports the same main numbers. Its wording differs in places:

  • It introduces absorption with "The key insight is that", as a premise of the method; the paper states it as a hypothesis that the gradient-norm results support.
  • It says SGTM required "7× more retraining" than "other unlearning methods"; the paper tests one unlearning method, RMU.
  • It says matching strict filtering's recovery suggests "genuine knowledge removal rather than superficial suppression". The paper's Appendix E adds the retain-matched comparison and the partial early recovery, which the post does not mention.

The labels that define what is removed

SGTM has no written principles or prompts. What it removes is set by labels: Wikipedia's article-topic classifier on the realistic task, and the language of each story on the synthetic one. The paper's argument is that the method should depend less on the quality of those labels than filtering does. It tests this with simulated labelers in the synthetic setting and with one real topic classifier on Wikipedia. It does not test labels produced from a written definition of harm, as in Constitutional Classifiers or the classifier-based filter of the August 2025 data-filtering post.

Limitations and open questions

The authors state these limitations (Section 6):

  • Scale. Models of 64M and 254M parameters, "orders of magnitude below" frontier systems; the authors call the results "proof-of-concept".
  • Proxy tasks and metrics. Spanish and Wikipedia biology stand in for CBRN knowledge, and loss stands in for capability. The biology share (about 4% of tokens) likely exceeds the real share of dangerous content.
  • Architecture. Dense transformers only; mixture-of-experts models are untested.
  • In-context attacks. Citing Shumailov et al. (2024), who argue that knowledge removal leaves models open to adversaries who supply the knowledge in the prompt, and O'Brien et al. (2025), who found data filtering susceptible to this, the authors "expect SGTM to behave similarly" and present it as one layer of a defense in depth.

The paper notes that SGTM's retain loss is higher than data filtering's across most settings, which it treats as a compute-efficiency penalty (Appendix D). The leakage trend covers models from 8M to 64M with a shrinking forget fraction. Absorption is supported by gradient norms in the synthetic setting only.

Outside review. The four public ICLR 2026 reviews (October and November 2025, anonymous, ratings 8, 6, 4 and 4) praise the clarity and the fine-tuning results. Their main objections are that every result is a loss, with no downstream evaluation of forgetting or retained capability; that the models are small; that the TinyStories label noise is random rather than realistic; that the paper compares only with filtering and earlier gradient routing, not with alternatives such as conditional training; and that the difference from earlier gradient routing was unclear in the submitted version. The meta-review was not retrieved.

Later work on absorption. The GRAFT paper (Ward, Hu et al., arXiv:2610.17062v1, October 2026), with Alex Cloud as an author, cites this paper as "suggestive evidence" for absorption, but reports that in its own exploratory toy experiments "gradient norms do not strongly separate by adapter as reported by Shilov et al." (Section 7). Those experiments are not reported in that paper.

Research questions

  1. Does SGTM's advantage hold on hazardous-knowledge benchmarks? All results are losses. Training models large enough to score above chance on WMDP and comparing SGTM with filtering on those questions would test whether loss differences become capability differences.
  2. Does leakage keep falling at larger scale with a fixed forget fraction? The 8M to 64M trend shrinks the forget fraction as models grow. Holding the forget share of parameters constant across sizes would separate the two.
  3. How does SGTM behave with a real CBRN classifier's errors? The Wikipedia labels are topic labels, not harm labels. Using a harm classifier's scores to define the labeled and unlabeled sets, at several thresholds, would test the method under the label noise it is designed for.
  4. How much of the fine-tuning resistance is localization rather than capacity loss? Appendix E shows SGTM matching the weak filter once general loss is equalized. Repeating the comparison with several seeds and with LoRA-style limited fine-tuning would show how stable the tie and the early lead are.
  5. Which unlearning baselines does SGTM outperform? Only RMU was tested. Comparing recovery against other post-training methods under the same fine-tuning budget would show whether the 7-fold gap is specific to RMU.
  6. Can one run yield both a useful full model and a well-ablated one at scale? Appendix C's relaxed configuration trades some removal for pre-ablation capability. Mapping that trade-off at a larger scale would show whether the two-model use is practical.

How it connects

  • Data filtering. The paper cites Enhancing Model Safety through Pretraining Data Filtering (August 2025) among the "cheap, imperfect filtering strategies" that pretraining scale forces, and compares SGTM with filtering throughout.
  • Gradient routing in RL. Reducing Reward Hacking by Localizing Bad Behavior Instead of Penalizing It (October 2026), with Alex Cloud as an author, applies gradient routing to reinforcement learning, routing monitor-flagged episodes into an adapter deleted after training. Its paper cites this one for unlearning, pretraining filtering and robustness to label noise, and reports weaker evidence for absorption in its own setting (see "Limitations and open questions").
  • Poisoning and scale. The paper contrasts its leakage trend with A small number of samples can poison LLMs of any size, which it cites for a roughly constant number of poisoned samples across model sizes.
  • Post-training defenses. The paper cites output classifiers as defenses that determined adversaries can circumvent, and the STACK attack on classifier pipelines; Constitutional Classifiers describes Anthropic's classifier defense.

References

  • Shilov, I., Cloud, A., Gema, A. P., Goldman-Wetzler, J., Panickssery, N., Sleight, H., Jones, E., and Anil, C. (2025). Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs. arXiv:2512.05648v1, submitted 5 December 2025. https://arxiv.org/abs/2512.05648 record
  • Shilov, I., et al. (2025, December 8). Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs [post]. Anthropic Alignment Science Blog. https://alignment.anthropic.com/2025/selective-gradient-masking/ record
  • Cloud, A., Goldman-Wetzler, J., Wybitul, E., Miller, J., and Turner, A. M. (2024). Gradient Routing: Masking Gradients to Localize Computation in Neural Networks. arXiv:2410.04332. record · 2 pages
  • Ward, J., Hu, S., Wong, A., Hu, N., Woodworth, B., Turner, A. M., and Cloud, A. (2026). Gradient Routing Localizes and Removes Unintended Behaviors in RL. arXiv:2610.17062v1. record · 2 pages
  • ICLR 2026 submission 'Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs': public reviews and author responses. OpenReview. https://openreview.net/forum?id=uMq2N8hJK4
  • Studies cited above only as the paper describes them (Shumailov et al., 2024; O'Brien et al., 2025; RMU and WMDP, Li et al., 2024; TinyStories; the ORES article-topic model) are listed in full in the paper.