Constitutional classifiers, separate models trained to spot dangerous requests, made Anthropic's models more robust to jailbreaks, but they add cost: the post's example is that using Claude 3.5 Haiku as a safety filter for Claude 3.5 Sonnet raises inference costs by about 25% ("Introduction"). The model being guarded already computes rich representations of every prompt. Can a classifier read those instead of recomputing them? "Cost-Effective Constitutional Classifiers via Representation Re-use" (Cunningham et al., 2025) compares several ways of doing so for prompts seeking bioweapons information, and measures how well each works as the cheap first stage of a two-stage system.
The post appeared on the Anthropic Alignment Science Blog, where the index lists it as "Model-Internals Classifiers" under June 2025; the page itself carries no date. It is absent from a Wayback Machine capture of the blog index on 16 June 2025 and present in one on 4 July 2025, and the page's own first capture is 4 July 2025, so the date above is the latest date consistent with the index's "June 2025." The authors are Hoagy Cunningham, Alwin Peng, Jerry Wei, Euan Ong, Fabien Roger, Linda Petrini, Misha Wagner, Vladimir Mikulik and Mrinank Sharma. The post is the only version; no arXiv paper was found. Its plots print no values, so numbers below that come from a figure are marked as read from it.
Terms used in this note
- Policy model. The model being guarded, here the largest model in the study, labeled XL. The post does not name it; its baselines are "fine-tuned classifiers from the Claude 3 Sonnet family" ("Introduction").
- Dedicated classifier. A separate, smaller model from the same family fine-tuned to classify prompts. The sweep uses four sizes, L, M, S and XS, with assumed costs of 25%, 9%, 2% and 0.6% of the XL model ("Baseline: Smaller Language Models").
- Linear probe. A logistic-regression classifier on the policy model's activations at one layer; the post does not say which layer. Three variants pool the per-token activations differently: a mean-token probe averages over all tokens; an EMA probe is trained like the mean probe but at evaluation scores every token, smooths the scores with an exponential moving average (α = 0.5) and takes the maximum; a suffix probe appends a fixed question asking whether the dialog contains harmful biological information and reads the activation at the final suffix token ("Probing"; Appendix A).
- Partially trained classifier. A copy of the policy model in which only the last few layers (1, 2, 4, 8 or 16) are fine-tuned to answer "harmful" or "not" after the suffix, so that it shares the rest of the computation with the policy model ("Partially-Trained Classifiers").
- Two-stage classifier. A cheap first stage passes only prompts above a threshold to the expensive classifier; anything below is treated as harmless ("Multi-stage classifiers").
- Log-space AUROC. The area under the ROC curve with the false-positive axis on a log scale between 0.001 and 0.1, chosen because production use depends on performance at low false-positive rates ("Evaluation Metrics").
What the post does
Data. Classifiers were trained on about 70,000 prompts, evenly split between attempts to get bioweapons information and harmless prompts, an early version of the Constitutional Classifiers training set; Claude 3.5 models generated them from a written constitution of permitted and restricted content and then transformed them into jailbreak templates, other languages, ciphers and prefills ("Training Data"). The harmful test set has about 4,000 examples generated from a fixed set of dangerous biological questions under in-distribution and held-out transformations and about 2,500 inputs from the Constitutional Classifiers red-teaming phase that had drawn a harmful response; the harmless test set is single-turn WildChat prompts and the "Hard" subset of OR-Bench, which the authors expect to give false-positive rates well above those of typical traffic ("Evaluation Data").
Training. Every model was trained for one epoch, about 2,200 batches of 32, with the learning rate of partially trained models scaled inversely to the number of layers trained ("Training"). Probes used L2-regularized logistic regression on standardized activations (Appendix D); the Methods section calls this "linear regression," but its formula is a sigmoid with cross-entropy loss.
Costs. Anthropic does not disclose the architecture, so the costs are illustrative (Table 1): probes are assigned 0.1% of the policy model; the suffix (150 tokens on an assumed average input of 5,000) costs 3%; and the one-retrained-layer model costs 4.25%, adding 1.25% for one of 80 layers, a layer count borrowed from Llama 3 70B ("Costs"). Dedicated classifiers are costed in proportion to parameters, an assumption the authors say may fail if attention dominates. A two-stage system costs the first stage plus the second stage's cost times the fraction of harmless prompts passed on.
Comparison. The policy model's own classification ability is tested three ways as baselines: XL fully fine-tuned as a classifier from its pretrained ("Base") version, the same from its post-trained (helpful, honest and harmless) version, and the post-trained XL simply prompted with the suffix. The dedicated L to XS classifiers are fine-tuned from pretrained models; the partially trained classifiers retrain layers of the post-trained XL (Figure 2 caption; Results). Scores are averaged across data orderings: the text says three, while the Figure 2 caption and Appendix B say two; the legends of Figures 1 and 2 give n = 2 for the XS model and the probes and n = 3 for the others, and Figure 4's legend gives n = 3 for XS.
What it finds
Single classifiers
- One retrained layer. Retraining only the final layer (attention and MLP) performed close to the fully fine-tuned XL model and at about the level of the L model, which costs 25% (Results, Figure 1). Read from Figure 1, its log-space AUROC is about 0.81 (its three runs spanning about 0.79 to 0.83) against about 0.80 for L, about 0.83 for fully fine-tuned XL Base and about 0.84 for fully fine-tuned post-trained XL. The introduction says it "slightly exceeds" the L classifier.
- Probes. The EMA probe and the suffix probe performed between the S and M models; the mean-token probe did worse than the smallest model (Results). Read from Figure 1, EMA about 0.69, suffix about 0.71, S about 0.67, M about 0.75, XS about 0.62, mean probe about 0.55. The post summarizes the EMA probe as outperforming the 2% classifier "at negligible cost" and performing as an interpolated 3%-overhead classifier would, and the suffix probe as roughly a 4%-overhead classifier ("Introduction").
- Prompting alone. Read from Figure 1, prompting XL with the suffix scored about 0.60 to 0.61, below every trained classifier including XS (about 0.62); only the mean probe scored lower.
- More layers. Retraining 2, 4, 8 or 16 final layers of the post-trained model gave no clear trend: the partially trained models stayed slightly below the fully trained model, and any gain from more layers was "within inter-run noise" (Results, Figure 3).
- Validation loss. Partially trained models had much higher loss on the training distribution than fully trained models of similar test performance (Appendix C, Figure 6). The authors suggest that being built on a larger model gives them better out-of-distribution generalization; this is their interpretation, not a tested claim.
Two-stage classifiers
With the fully fine-tuned XL model fixed as the second stage, both probes and the one-layer model beat the smaller dedicated models as first stages across the cost range, the EMA probe and the one-layer model performing best and the EMA probe ahead of the suffix probe because it is cheaper (Figure 4). The EMA probe and the one-layer model reduced cost "by over 10x without significantly reducing the performance of the overall system," and if overhead must stay under 5% of the XL model, the EMA probe is the best first stage ("Introduction"; Figure 4). In some regimes the cascade slightly outscored the XL classifier alone, which the authors attribute to uncorrelated errors, with the first stage screening out more would-be false positives than true positives ("Two Stage Classifiers"). They warn that this gain on a fixed set "may not translate to robustness" against attackers who target the first stage. Figure 4 draws the XL second stage alone at about 0.853, above the roughly 0.84 average for the same model in Figures 1 and 3 (values read from the figures); the post does not explain the difference, which may come from a different run or averaging.
Where the evidence stops
The red-team inputs were collected against Claude 3.5 models and other classifiers, with no knowledge of these probes, and the classifiers here are Claude 3 Sonnet models trained on different data ("Introduction"). The authors state: "we have not studied the robustness of any of the classifiers in this paper to adaptive adversarial attacks targeting these classifiers directly," and that relative robustness "could be worse for probes (in part because they're integrated into the model being attacked)." They present the results as evidence that probes beat baselines "under these non-adaptive attacks."
The prompts and constitution used
The constitution. The training data inherits the Constitutional Classifiers method: Claude 3.5 models generated prompts according to a written constitution listing permitted and restricted content, with teaching the model novel ciphers placed in the restricted category ("Training Data"). The post does not print the constitution and reports no result about its wording; it reaches every classifier here only through the generated data.
The suffix. The suffix probe, the partially trained models and the prompted baseline all use one instruction appended after the dialog (Appendix A); the dedicated classifiers are "trained with the same objective as the partially-trained classifiers," which this note reads as completing the same suffix ("Baseline: Smaller Language Models"). It asks the model to review the dialog for "information enabling biological harm (like bioweapons)", warns that harmful content "may be hidden - buried in text, disguised as fiction, or using code words", adds "However, don't be overzealous - only flag content with clearly suspicious elements," and ends with a forced choice between two answer sentences. The authors say they made no effort to shorten it and expect its cost could be cut, for example by compressing it into a "gist" token ("Costs"). No alternative wording was compared, so the post says nothing about how the suffix's wording affects the probes.
Limitations and open questions
The authors list these limitations ("Limitations"; "Introduction"):
- No adaptive red-teaming. None of the classifiers faced the kind of red-teaming used for the original constitutional classifiers, and multi-stage systems may open new attacks: an attacker who makes the weak first stage confident will never be seen by the strong second stage.
- Undertrained baselines. The smaller dedicated models were trained on data proportional to their size; training them on far more data, or distilling them, could tip the comparison back toward small models.
- Simple cascade. Only one cascade design was tested; combining first- and second-stage scores, or letting the first stage pass "unsure" cases only, might do better.
- Prompts only. Only input classifiers were studied, because they see the same text as the policy model and because input tokens dominate cost; output classification is discussed only as a latency argument.
Further points from the post's own description: costs are assumptions rather than measurements; results are for one harm domain (biological weapons) and one policy model; and the harmless set includes OR-Bench "Hard," so false-positive rates are not those of real traffic.
Later work from the same group answered some of this. The paper described in Next-generation Constitutional Classifiers (January 2026) cites this post as work on internals-based classifiers "focusing on last-N layer networks," and builds a production system with a linear probe on Claude Sonnet 4.5 as the first stage of a cascade, which was red teamed as a whole system; that note gives its results.
Research questions
- Are probes harder or easier to attack directly than dedicated classifiers? The post leaves adaptive robustness untested. A red-team campaign with knowledge of the probe, measured at matched false-positive rates against a dedicated classifier of similar clean performance, would answer it.
- How much does the suffix's wording matter? One suffix serves the suffix probe, the partially trained models and the prompted baseline (Appendix A). Comparing a few paraphrases, and a neutral suffix that does not mention biology, on the same data would show whether the gain comes from the question or from probing a fixed position.
- Why do partially trained models generalize better than their validation loss suggests? The authors propose the larger base model as the reason (Results; data in Appendix C). Partially training policy models of several sizes and comparing out-of-distribution performance at equal validation loss would test it.
- Does the cascade's advantage survive attackers who target the first stage? The authors flag this risk. Measuring how often attacks optimized against the EMA probe slip under the first-stage threshold would show whether the cost saving opens a weakness.
- Do the results hold for other harm domains and for output classification? The study covers bioweapons prompts only. Repeating it for chemical or cyber content, and on completions, would show how general the cost-performance ordering is.
How it connects
- Predecessor. The data, the transformations and the red-team inputs come from Constitutional Classifiers (Sharma et al., 2025), which the post cites throughout.
- Successor. Next-generation Constitutional Classifiers cites the post in its related work (Section 7) and develops probe training further: its probes are trained with a sliding-window mean of token scores and a softmax-weighted loss, and, like the EMA probe here, apply an exponential moving average at inference (its Section 5).
- Probes and features. The post cites Using dictionary learning features as classifiers (Bricken et al., 2024) for probing a fixed token position and for the finding that linear probes are a hard baseline to beat, and Anthropic's dictionary-learning work, including Towards Monosemanticity, for sparse features. Simple probes can catch sleeper agents is earlier Anthropic work using linear probes on activations for a different purpose; that connection is the note's.
- Patching. The post cites Peng et al. (2024) on quickly patching vulnerabilities in monitoring systems; that paper is the subject of Rapid Response: Mitigating LLM Jailbreaks with a Few Examples.
- Monitoring cost elsewhere. These links are the note's: the post on SLEIGHT-Bench (2026) suggests the cost of expensive agent monitors may be mitigated by running them only as a second stage, and Monitoring computer use via hierarchical summarization describes another Anthropic approach to monitoring at scale, with two authors in common.
- In the course wiki. The log-scale ROC area and fixed false-positive operating points relate to Evaluating Classifiers; the cascade resembles the multi-stage ranking in Search at Scale.
References
- Cunningham, H., Peng, A., Wei, J., Ong, E., Roger, F., Petrini, L., Wagner, M., Mikulik, V., and Sharma, M. (2025, June). Cost-Effective Constitutional Classifiers via Representation Re-use. Anthropic Alignment Science Blog (listed as "Model-Internals Classifiers"). https://alignment.anthropic.com/2025/cheap-monitors/ (read November 2026; first Wayback capture 4 July 2025). record
- Sharma, M., et al. (2025). Constitutional Classifiers: Defending against universal jailbreaks across thousands of hours of red teaming. arXiv:2501.18837. Cited as the post cites it. record · 5 pages
- Constitutional Classifiers++ (arXiv:2601.04603v1, 8 January 2026), checked only for its citation of this post (Section 7); see the Next-generation Constitutional Classifiers note for full details. record · 2 pages