Modular Pretraining Enables Access Control

Roland, Cubuktepe, Martinez et al. (AE Studio, with Anthropic) propose gradient-routed auxiliary modules (GRAM), which train each dual-use domain into small removable modules during pretraining. On an 800M-parameter model with virology, cybersecurity, nuclear physics and Lisp code as the removable domains, deleting modules matched separately trained data-filtered models on held-out loss and resisted a small fine-tuning attack about as well, at one-fifth of filtering's training compute; a LoRA baseline did about as well, and GRAM's advantages over it (combining modules, half-labeled data) come from single experiments.

Ethan Roland, Murat Cubuktepe, Erick Martinez et al. (AE Studio, independent, Anthropic) · 2026-07-08 · original

A developer that wants to offer a dangerous capability, such as advanced virology, only to vetted users has two kinds of tool. It can control behavior (refusal training and classifiers), which leaves the knowledge in the weights where a jailbreak can reach it, or it can control knowledge, by filtering the training data so the model never learns the domain. Filtering yields one model with one fixed set of capabilities; serving several sets means training several models. "Modular Pretraining Enables Access Control" asks whether one pretraining run can produce a model whose dangerous domains can each be switched off afterward, with each configuration behaving like a model trained without that domain's data (Section 1).

The work exists in several documents. The paper, arXiv:2607.08077v1 (submitted 9 July 2026, 34 pages including appendices A to N), is the version of record and the source of every location below unless marked "post". The Alignment Science Blog post of 8 July 2026 summarizes it, and the shorter anthropic.com post An off switch for dual-use knowledge presents it for a general audience. The paper was presented as a poster at ICML 2026 (session of 9 July 2026); the authors' project page labels it a spotlight, which the ICML page does not state. The abstract on the ICML virtual-site page differs from the arXiv version: it names the method "gradient-routed mixture-of-experts (GR-MoE)", reports scaling "from 48M to 2B parameters", and says the method "preserves capabilities better than LoRA". The arXiv version calls it GRAM, scales to 5B, and reports GRAM and the LoRA baseline as comparable in the main setting (Section 5). This note did not retrieve a proceedings PDF and follows arXiv v1. Code is at github.com/agencyenterprise/modular-pretraining (last commit on the release date, d07d62e, 8 July 2026); the authors' project page also lists a released dual-use papers dataset on Hugging Face and says model weights are "coming soon".

The authors are Ethan Roland, Murat Cubuktepe and Erick Martinez (equal contribution), Stijn Servaes, Keenan Pepper, Mike Vaiana, Diogo Schwerz de Lucena and Judd Rosenblatt (all AE Studio), Addie Foote (independent), and Cem Anil and Alex Cloud (Anthropic). Anthropic provided compute for the scaling experiments (Acknowledgements). The post says the method "has not been applied to production models at Anthropic".

Terms used in this note

  • Core and auxiliary data. The core dataset is general-purpose text. Each auxiliary dataset is one capability to be controlled (a story topic, or a dual-use domain).
  • Capability profile. The set of auxiliary capabilities a deployed model should have. Its retain set is the auxiliary domains kept; its forget set is those removed (Section 2).
  • Compute ratio. The paper's metric. For a model and a dataset, it is the fraction of the all-data baseline's training run at which the baseline reached the same validation loss. 1.0 matches the fully trained baseline; 0.5 means the model is as good as the baseline was halfway through training. Higher is better on core and retain data; lower means more removal on forget data (Section 2, Appendix M).
  • Elicited forget. The forget compute ratio after an attacker fine-tunes the model on a fixed sample of forget data and the best checkpoint is kept (Section 2).
  • Data filtering. Training a separate model from scratch on core data plus only the retained domains: the paper's "gold standard" for removal.
  • FT-LoRA and FT-Full. Branched baselines: train a core-only model, then fine-tune one low-rank adapter (FT-LoRA) or one full copy (FT-Full) per domain on that domain mixed with core data.
  • MaxEnt. A post-hoc unlearning method that makes the model's output distribution as uncertain as possible on forget data (Yuan et al., 2025).

What the paper does

The method

GRAM widens the MLP layer of every transformer block with small auxiliary MLP modules, one per auxiliary domain, beside a core MLP the size of a normal one. There is no learned router: which modules run depends on the label of the training batch (Section 3, Figure 1).

  • Auxiliary batches. The core MLP and that domain's module run in the forward pass. The module is always updated; the shared parameters are updated only with probability pas, the "auxiliary spread". Lower values isolate the domain more strongly.
  • Core batches. The shared parameters are updated. With probability pcr, the "core robustness", one randomly chosen auxiliary module also runs and is updated, so that core performance does not depend on which modules are present at inference.
  • Inference. A profile is served by keeping its modules and deleting the rest.

Each parameter group has its own AdamW optimizer, following Shilov et al. (2025). The paper presents GRAM as combining domain-specific modules (DEMix), label-dependent gradient updates (gradient routing, Cloud et al., 2024) and separate optimizers (Section 1).

Three settings

  • SimpleStories. A synthetic corpus of about 2 million children's stories in 48 topics. The first four topics alphabetically (A Deadline or Time Limit, Alien Encounters, Bygone Eras, Cultural Traditions) are auxiliary; the other 44 are core. Models have 26M parameters and 8 layers, three seeds per method, pas = 0.3 and pcr = 0.5 (Section 4, Appendix A).
  • Realistic dual-use data at 800M parameters. Core is 16B tokens of FineWeb-Edu web text, non-Lisp code from The Stack and general arXiv papers. The auxiliary domains are virology papers (Europe PMC), cybersecurity papers (arXiv cs.CR, cs.NI and cs.OS), nuclear physics papers (OSTI and arXiv) and Lisp code, used as a stand-in for a niche or proprietary codebase: about 40M tokens each, 1% of the core in total (Section 5, Appendix B, Table 2). General arXiv papers sit in the core, processed like the auxiliary papers, so that loss differences reflect domain content rather than formatting. Each GRAM module has 49.2M parameters, 10% of the baseline MLP width; pas = 0.5 and pcr = 0.2. Labels are assigned per dataset, not per document.
  • Scaling. Seven sizes from 50M to 5B parameters, with Chinchilla-optimal data and auxiliary data at 1% of core. Only one profile is reported: core and virology retained, the other three domains removed. Three seeds below 2B; one run at 2B and at 5B (Section 6, Figure 7).

The attacker

For elicited forget in the realistic and scaling settings, each model is fine-tuned on 512 sequences of 1,024 tokens (about 0.5M tokens) from each forget domain, about 1.3% of a domain's training data at 800M, for up to 200 epochs with early stopping, and the best validation loss is reported (Appendix B). The sample size is the same at every model size. The authors call this "a limited adversary" and expect partial recovery "even from data-filtered models" with more data or compute (Section 2). On SimpleStories the attack is 75 steps on 128 sequences per forget category (Appendix A).

What is held equal

All methods except filtering use the training compute of one baseline run; filtering trains five models. GRAM and FT-LoRA have more parameters per forward pass than the baseline (1.1 times the MLP parameters when a module is active, 1.4 times in total), so their training is shortened to match baseline FLOPs (Figure 4 table, Appendix I). The metric throughout is held-out loss; no downstream task or benchmark is evaluated.

What the paper finds

One run approximates five filtered models on SimpleStories

With Alien Encounters retained, filtering and GRAM both score 0.99; a removed topic, A Deadline or Time Limit, falls to 0.65 under filtering and 0.61 under GRAM (Section 4, Figure 2). Averaged over profiles, GRAM's retain score is 0.95 against filtering's 0.96, and core 0.94 against 0.96 (Section 4). On forget data, GRAM scores 0.77 and filtering 0.78; after the fine-tuning attack, 0.86 and 0.87. MaxEnt's forget score of 0.84 rises to 0.98 after the attack. DEMix, without a shared core, scores 0.25 on core and 0.18 on retain (Appendix A, Table 1; means of three seeds). Increasing the number of auxiliary topics from 4 to 20, with their share of training tokens fixed at 20%, kept core near 0.88 and retain at about 0.89 to 0.91, with forget at about 0.65 to 0.74 and elicited forget at about 0.71 to 0.84, both rising modestly as each topic got less data (Section 4, Appendix D, Figure 3).

Realistic dual-use data at 800M

Averaged over the five profiles (all four domains removed, or one of the four kept), compute ratios were (Figure 4, printed labels; three seeds, 90% confidence intervals):

Method Core Retain Forget Elicited forget
Filtering (5 runs) 0.99 0.82 0.53 0.58
GRAM 0.96 0.97 0.54 0.63
FT-LoRA 0.95 0.98 0.51 0.56
FT-Full 0.96 0.88 0.52 0.57
MaxEnt 0.89 1.01 0.46 0.91

The authors summarize that GRAM and FT-LoRA "approximate data filtering well" while MaxEnt's removal "is mostly reversed by finetuning" (Section 5). After the attack GRAM's forget score (0.63) is above filtering's (0.58) and FT-LoRA's (0.56); the paper describes the methods as reaching "roughly equal" losses and does not discuss this difference. The authors add that their setting, with little auxiliary data and one epoch, "is favorable to low-rank adaptation", and that FT-LoRA may not stay competitive with longer fine-tuning (Section 5). Their explanation for GRAM's higher retain score than filtering is that GRAM trains on all four domains and benefits from transfer between them (Section 8).

Where GRAM and FT-LoRA differ

  • Combining modules. With all four modules switched on, a configuration never used in training, GRAM's virology score stayed at the level of the virology-only configuration, while summing four LoRA adapters lowered FT-LoRA's scores in every category (Section 5, Figure 5, which prints no values). Only these two configurations are shown; the paper's count of 2^(N−1) profiles, 16 here, assumes that every combination works.
  • Half the labels missing. With 50% of training data unlabeled, filtering and FT-LoRA treated unlabeled data as core, and GRAM ran every module on unlabeled batches. For the configuration with all domains removed, forget scores were 0.60 ± 0.01 for GRAM against 0.85 ± 0.01 for both filtering and FT-LoRA, with core at 0.96 against 0.99, and elicited forget 0.76 against 0.85 and 0.86 (Section 5, Figure 6). GRAM with half the labels did not reach filtering with full labels, whose forget and elicited forget scores were 0.54 and 0.67 (Appendix G, Figure 13). This experiment used a 400M model, and the authors "find it necessary" to change GRAM's architecture for it (a core MLP at 95% of the baseline's width and modules at 1.25%) (Appendix G). They attribute the result to "absorption", the observation from earlier gradient-routing work that once labeled examples make a module specialize, similar unlabeled examples update it too, and leave its direct investigation to future work (Section 8, Appendix G).

Scale

At every size the three methods stay near the baseline on core and virology, and the gap on the removed domains widens with size: forget scores fall from about 0.7 at 50M to about 0.4 at 5B, and elicited forget from about 1.05 to about 0.45 (read from Figure 7, which prints no values). At the smallest size, then, the fine-tuning attack restored the removed domains to baseline level for all three methods. The paper "speculate[s]" that at small scale most of the baseline's gain on any domain comes from general language structure, while at larger scale more of it is domain-specific (Section 6). It reads the trend as showing that "data filtering itself scales favorably" (Section 9). The post adds a caveat the paper does not state in these words: the values are relative to a baseline, and "In absolute terms, the bigger models are still better at the forgotten capabilities." The falling elicited score is also measured with a fixed 0.5M-token attack, which the paper notes is a shrinking fraction of training data as models grow (Section 6).

GRAM's retain score at 5B is above the trend. The authors trace this to a change in how gradients were accumulated: the 5B run used a method that keeps each module's gradients separate, while smaller GRAM runs did not, and the auxiliary-factor setting tuned to offset that was not lowered at 5B. A single-seed comparison at 200M shows the change raising GRAM's retain score from 0.75 to 1.11 with forget almost unchanged (0.70 and 0.69). The 400M and 800M GRAM numbers are "likely slight underestimates" of retain (Appendix H, Table 4).

The isolation dial

On SimpleStories, raising pas from 0 to 1 raised core from 0.90 to 0.97 and retain from 0.88 to 1.04, and raised forget from 0.69 to 0.89 (Appendix F, Figure 11): the setting trades removal for general quality. The main runs use 0.3 (SimpleStories) and 0.5 (realistic data).

Post and paper

The Alignment Science Blog post reports the main results in the paper's terms. Its scaling paragraph says that "bigger models fall further behind the all-data baseline on virology when the capability is removed". In the paper's scaling experiment virology is the retained domain, and the removed domains are the other three (Section 6; Figure 7 labels "Retain (Virology)" and "Forget (Non-Virology)"). The post's figure of "about 0.25%" per domain matches Table 2.

The labels that define what is removed

GRAM has no written principles or prompts. What a module holds is set by data labels, here assigned by source: a paper from Europe PMC's virology literature is virology, arXiv cs.CR is cybersecurity (Appendix B). The paper tests robustness to missing labels (Section 5) but not labels produced by a classifier applied to a written definition of hazardous content, as in Enhancing Model Safety through Pretraining Data Filtering, and the authors list label errors that are not random as future work (post, Discussion).

Limitations and open questions

The authors state (Section 8):

  • Entangled capabilities. Where a dangerous domain overlaps heavily with a wanted one (virology and biology), "clean separation may be impossible regardless of method"; the post adds that filtering itself might then be ineffective (post, Discussion).
  • Scale and production. Trends hold to 5B; frontier scale and production training are untested, and implementing GRAM "would require significant effort and introduce complexity that frontier AI developers may prefer to avoid".
  • Why GRAM differs from FT-LoRA. No account yet of the composability result or of absorption; the post calls these "two isolated experiments".
  • Post-training. Interaction with instruction tuning or RLHF, which "could either reinforce or erode" the separation, is unknown.
  • Loss only. Cross-entropy is "a strong predictor" of downstream performance "but may not give a complete picture".

This note adds, from the paper's own setup: the scaling and partial-labeling results rest on one profile each, the 2B and 5B points are single runs, and the attacker is fixed in size.

Comments. On the Alignment Forum crosspost (9 July 2026), a commenter asked whether the method stays useful once models can re-derive missing knowledge; Mike Vaiana, an author, replied that the question applies equally to data filtering and argued that both remain useful: for closed models the developer could monitor knowledge supplied in context and flag it before the model gives dangerous instructions, and a model kept from learning a domain is "better or equal to" one forced to re-derive it in context, which needs "huge amounts" of supplied material.

Research questions

  1. Do loss differences become task differences? Every result is held-out loss. Evaluating removed and retained configurations at the 5B scale on domain question sets or agentic tasks would show whether a forget score of about 0.4 means a practical loss of capability.
  2. How does recovery grow with the attacker's budget? The 0.5M-token attack is fixed. Curves of elicited forget against fine-tuning tokens, for GRAM and filtering at several sizes, would show whether the widening gap survives a stronger attacker.
  3. Do all 16 combinations work? Only "virology only" and "all four" were compared. Scoring every profile from one 800M run would test the composability claim directly.
  4. Does the partial-labeling advantage hold with the main architecture and realistic label errors? It was shown once, at 400M, with modified module sizes and random unlabeling. Repeating it with classifier-produced labels and the standard architecture would test it.
  5. Does the separation survive instruction tuning and RLHF? Post-training the GRAM model, then ablating modules and re-running the attack, would answer the authors' open question (v).
  6. How entangled is virology with general biology? Measuring the removed-virology configuration's loss on general biology and medicine text would put a number on the entanglement the authors name.

How it connects

References