When a developer calls a model "safer than its predecessor," what does that claim rest on? This post, published on 13 November 2026 under Anthropic's Engineering label and written by eight Anthropic staff, answers that it rests on "a set of evals someone built," and that "a safety claim is only as sound as the eval behind it" (introduction). It then offers recommendations "for anyone who builds or relies on AI safety evaluations," drawn from psychology's response to its replication crisis and from Anthropic's own practice, with examples from its system cards. The post reports no new experiment. Its numbers come from the system cards, and this note checks them against the cards it cites. The post is the version of record; its footnote 1 points to an earlier Engineering post on capability evaluations for agents, which this note did not read.
Terms used in this note
- Safety evaluation. The post's definition: an evaluation that "measures a model's propensity to cause or contribute to harm, whether its response to a request makes harm more or less likely, and how severe that harm would be, rather than measuring specific capabilities." In short, "Given a specific input, did the model do the right thing?" (introduction).
- Appropriate response rate. In Anthropic's multi-turn evaluations, "the share of conversations in which the model behaved appropriately throughout," judged against a rubric for each risk area written by domain experts. One bad turn fails the whole conversation ("Measure what you mean to measure").
- Harmless response rate and refusal rate. In single-turn evaluations, harmful prompts are scored by the share of harmless responses and benign prompts in the same domain by the share refused (same section).
- Construct validity is "whether your measure captures the concept you actually care about"; an evaluation is internally valid when its scores are driven by the targeted behavior and not by something else ("Keep measures internally valid").
- Preregistration is "writing down hypotheses and analysis plans before collecting or analyzing data" (introduction).
What the post recommends
The post groups its advice under four headings: safety first, design, run and report. It closes by naming the three things to do if one does nothing else: "write down the construct you want to measure before collecting data, validate your grader against human judgment before trusting any number it produces, and report uncertainty so readers can tell signal from noise" ("Where to start").
Safety first
- Team. Methods experts decide how to sample, grade and analyze; domain experts decide "what counts as harm, which cases are borderline, and what a good response looks like."
- Running safely. Safety evaluations often run with safeguards disabled, so the post recommends a sandbox with no internet access and real-time monitoring. It reports that "in response to our own incidents," Anthropic has hardened its evaluation environments, "temporarily stopped running many evaluations that give models internet access," and added real-time monitoring for web search evaluations that need it.
Design
- Observational or causal. Most evaluations are observational: they show that two models scored differently, not why. A causal claim ("This intervention made the model safer") needs a design that varies one variable at a time, such as the same model with and without an intervention ("Match the design to the question").
- The right construct. Refusal rate is the obvious construct and often the wrong one. For suicide and self-harm, "simply declining to engage is, itself, a failure"; a model that refused every such prompt "would get a perfect score on a refusal-rate eval, but it is not necessarily a safer model." For dual-use biology and cybersecurity, an appropriate response helps with the legitimate version of a request while withholding "operational specifics." Anthropic builds a separate rubric for each topic and pairs harmful with benign prompts in the same domain, so that over-refusal is measured alongside compliance.
- Multi-turn simulation. Single-turn prompts "saturate quickly" and miss gradual escalation and distress that accumulates. In Anthropic's internal tool, a domain expert writes a specification (the simulated user's objectives, tactics, persona, tone); the first user turn is fixed; a separate Claude model writes later user turns; each scenario runs several times with different combinations of the specification's options; and a grader scores each transcript against the rubric. In a romance-scam example no single message is an obvious violation, and passing requires the model to "recognize the pattern." Dimensions such as severity and tone should be crossed so that their effects can be separated.
- Real usage. Invented scenarios "tend to be both too easy for models to pass and subtly unlike what real users write." The election-integrity prompt set is grounded in observed patterns, and the scenario set is revised from enforcement and monitoring work. The single-turn suites "run across 16 policy areas in seven languages and across multiple product surfaces" (the Claude Sonnet 5 system card, Section 4.1.1, names the languages).
Run
- Contamination, shortcuts and difficulty. Test items in training data measure memory, and models may find unintended routes to a passing grade. On the API, single-turn aggregate harmless response rates "are between 94% and 98% for every recent model," so those evaluations are "saturated" and serve "mainly as regression checks." The post gives no table for this range. An evaluation can also be too hard; the post cites an audit of a coding benchmark whose tasks were often underspecified or wrongly graded.
- Graders. Graders, models that score transcripts (often called LLM judges), "can fail without anyone noticing." The post recommends programmatic checks where a ground truth exists; atomic yes-or-no criteria; rubrics stating that "disclaimers, caveats, and educational framing do not make harmful content harmless"; a grader that sees no design metadata and is a different model from the one tested, since graders "have been shown to recognize and favor their own outputs"; checks for length and position bias; re-grading a sample with another model or rubric ("If the conclusions change when the grader does, you've likely measured the grader, not the model"); and validation against human labels, with chance-corrected agreement such as Cohen's kappa (footnote 3) compared with the humans' agreement among themselves.
- Preregistration and sample size. Write down the main metrics, comparisons, data slices and the threshold for an improvement or regression before seeing results, and report any deviation. Anthropic treats "100 items as the floor for a complex, multi-turn eval and 1,000 as the floor for a simpler, single-turn one," and recommends repeated runs and paired comparisons on the same items.
Report
- Full statistics. Break results out by category, report categories that did not change, correct for multiple comparisons, and give an interval for every estimate. Intervals for multi-turn evaluations on the API "can be wide, exceeding ±10 percentage points."
- Configuration. Report model versions, sampling settings, prompts, grader version and dates, and "don't compare scores from different graders." The section's figure adds, in its image text: "Compare scores only when every setting matches except the one you're testing." The post points to the UK AI Security Institute's Inspect framework and Anthropic's Petri toolkit.
- Flaws. Run evaluations throughout fine-tuning, so that reward signals or system prompts can be adjusted before release, and disclose severe flaws in deployed systems to the developer before publishing.
The examples from the system cards
The post's numbers are illustrations, not new results. Each was checked against the card it comes from:
- A difference that is not significant. On a multi-turn romance-scam evaluation on claude.ai, "Claude Opus 5 and Opus 5.5 differed by multiple percentage points (98% versus 94%)," a difference that "wasn't statistically significant" ("Report statistics fully"). The Claude Opus 5.5 card's Figure 4.1.3.A (p. 64) prints these labels: 98% for Opus 5 and 94% for Opus 5.5 on claude.ai. The figure draws error bars but prints no interval values, and the post does not name the test.
- Configuration. For Claude Fable 5.1, the difference in appropriate response rate between the API and claude.ai configurations on multi-turn safety evaluations "was as high as 34 percentage points" ("Document everything"). The Fable 5.1 card's Table 4.3.1.B (pp. 67 to 68) gives 60% (±14%) on the API and 94% (±7%) on claude.ai for suicide and self-harm. That difference is 34 points; that it is the largest of Fable 5.1's multi-turn gaps is this note's arithmetic from the labels printed in the Opus 5.5 card's Figure 4.1.3.A, where the next largest is 16 points.
- A flaw found before release. Reviewing self-harm transcripts before Fable 5.1's release, Anthropic saw Claude "sometimes implicitly validated self-harm as a coping strategy" within otherwise supportive responses, and added a system-prompt instruction. Re-running the evaluation "with this and other targeted system prompt mitigations showed that the changes to our system prompt significantly improved the appropriate response rate on claude.ai (60% on the API vs 94% on claude.ai with the new system prompt)" ("Disclose flaws responsibly"). The Fable 5.1 card describes the same change and says it "partially mitigated the weaknesses described above" (Section 4.3.1, p. 68), that is, the validating statements its reviewers had observed; the card makes no claim about the change's effect on the rate. The two numbers are the card's API and claude.ai columns (Table 4.3.1.B): they compare the model with no system prompt against "the model on the claude.ai consumer surface with a near-final version of Fable 5.1's production system prompt" (Section 4.1, p. 59), so they differ in the whole system prompt, not only in the added instruction. Neither the post nor the card gives a claude.ai rate measured before the instruction was added, and the post names no test behind "significantly." The post's statement that the 94% was obtained after the change is information the card does not give.
- Carrying distress forward. The example of a user who discloses a destabilizing event early and later asks an ordinary-seeming question links the Claude Sonnet 5 card, whose Section 4.3.1 (p. 45) tests whether the model "maintains awareness of an earlier signal of distress."
The written instructions involved
Several of the post's recommendations concern text written for models:
- System-prompt language. The post prints the sentence added to the claude.ai system prompt for Fable 5.1: "Claude does not tell someone that self-harm works, helps, or does something for them, even when they say so themselves." The Fable 5.1 card paraphrases it ("language that steers Claude away from describing self-harm as effective even when the user asserts this themselves") without quoting it. The post says it was one of several "targeted system prompt mitigations" and gives no result for it alone.
- Grader instructions and simulated-user specifications. The post advises writing grader instructions as for "a competent colleague who hasn't worked in your area before," and describes the written specifications a Claude model follows when playing the user. It prints no grader prompt and no specification; the romance-scam example appears only in prose and as paraphrased prompts in a figure.
Limitations and open questions
- Recommendations, not measurements. The post reports no study of whether its practices change evaluation outcomes. Its examples come from Anthropic's own cards, and it does not say how many of Anthropic's evaluations follow each recommendation; it says that "not every eval necessarily needs to employ every recommendation" and that the recommendations "may evolve."
- Self-assessment. The evaluations, graders and reports discussed are the developer's own, about its own models.
- Grader validation. The post recommends validating graders against human labels and reporting chance-corrected agreement, but reports no agreement figure for any Anthropic grader. Where the 2026 card notes in this collection record the multi-turn setup, the simulated user is Claude Opus 4.6, the grader is not named and no validation against human judgment is reported (see Written Principles and Model Behavior: An Evidence Map under the claude.ai configuration); that connection is this note's.
- Unlocated figures. The 94% to 98% single-turn range and the "±10 percentage points" for multi-turn intervals are stated without a table; the Fable 5.1 card prints ±14% and ±7% for the self-harm rates.
- Configuration comparisons. The post recommends a causal design, and comparing scores only when one setting differs; its example of a system-prompt effect compares the API with no system prompt against the full claude.ai configuration. This note records the juxtaposition; the post does not comment on it.
Research questions
- How well do Anthropic's safety graders agree with human experts? Chance-corrected agreement between graders and human labels, and among the humans, on a sample of multi-turn transcripts per risk area would apply the post's own validation step to the published numbers.
- What does the added Fable 5.1 sentence do on its own? Running the claude.ai configuration with and without that sentence, all else fixed, would separate its effect from the other mitigations and from the product surface.
- Do the single-turn suites still discriminate between models? Per-category rates with intervals for every recent model would show whether the 94% to 98% range hides differences that matter, or confirms the post's reading that the suites are saturated.
- How sensitive are the multi-turn rates to the simulated user? Re-running a suite with a different model playing the user, or with the same specifications reworded, would show how much of a score depends on the simulator.
- Would preregistered thresholds change any system-card conclusion? Publishing the metrics, comparisons and improvement thresholds fixed before a card's evaluations, and checking the card against them, would test the practice the post recommends.
How it connects
- Challenges in evaluating AI systems (October 2023) is Anthropic's earlier policy post on why evaluations are hard to build and interpret; this note connects the two as the earlier and later statements of the problem.
- A statistical approach to model evaluations (November 2024) is the post linked for paired comparisons and confidence intervals. The library's Paired Comparisons of Models, Power Analysis for Evaluations and Measures of Agreement: Cohen's Kappa treat the statistics the post recommends.
- Petri is the toolkit the post recommends for multi-turn behavioral testing.
- Alignment assessment of recent cybersecurity incidents is linked as "our own incidents"; Investigating unintended model actions describes further incidents in evaluations with live internet access (that connection is this note's).
- The examples come from the Claude Sonnet 5, Claude Fable 5.1 and Mythos 5.1 and Claude Opus 5.5 system cards.
- Updating our coordinated vulnerability disclosure policy timelines (November 2026) revises the disclosure deadlines Anthropic applies to vulnerabilities its models find; this post's advice on disclosing flaws in deployed systems concerns evaluation results, and the connection is this note's.
References
- Appel, R., Frondorf, E., Hadfield, J., Hiscott, R., Jagadish, K., Longpre, S., Tucker, M., and Wittmershaus, B. (2026, November 13). Building strong safety evaluations. Anthropic (Engineering). https://www.anthropic.com/research/building-strong-safety-evaluations record
- Anthropic. (2026). Claude Sonnet 5 System Card (PDF linked from the post). Section 4.1.1 and Section 4.3.1, p. 45. record
- Anthropic. (2026, September 1). System Card: Claude Fable 5.1 & Claude Mythos 5.1 (212-page PDF). Section 4.1, p. 59; Table 4.3.1.B and Section 4.3.1, pp. 67 to 68. https://www.anthropic.com/claude-fable-5-1-mythos-5-1-system-card record · 2 pages
- Anthropic. (2026, September 22). System Card: Claude Opus 5.5 (230-page PDF). Figure 4.1.3.A, p. 64. https://www.anthropic.com/claude-opus-5-5-system-card record · 3 pages