Anthropic's Pilot Sabotage Risk Report

A 67-page practice "affirmative case" under Anthropic's Responsible Scaling Policy, with evidence to 20 August 2025, arguing from capability limits, an alignment audit and partial monitoring that the risk of Claude Opus 4 (and Opus 4.1) taking misaligned autonomous actions that contribute significantly to later catastrophe is "very low, but not completely negligible"; published with an internal Alignment Stress-Testing review and an external METR review, both of which agree with the bottom line while questioning parts of the evidence, especially for the alignment claim.

Samuel R. Bowman, Misha Wagner, Fabien Roger and Holden Karnofsky (Anthropic); reviews by Daniel M. Ziegler and Evan Hubinger (Anthropic) and by METR · 2025-10-28 · original

Anthropic's Responsible Scaling Policy (RSP) commits the company, for future models past a capability threshold it had not yet reached, to write "an affirmative case" that identifies the most relevant risks from models pursuing misaligned goals and explains how they have been brought to acceptable levels. In October 2025 Anthropic published a practice version of such a case for the model it had studied most closely, Claude Opus 4. The post introducing it says the authors had "never prepared a complete affirmative case of this kind" and were not aware of such a document from any other frontier developer. The report asks one question: how much risk is there, as of summer 2025, that Anthropic's own models take misaligned actions that substantially contribute to a later catastrophe? It answers "very low, but not completely negligible." It matters to this collection as the first of Anthropic's risk reports, a genre in which the developer argues about its own models' safety in writing and invites named reviewers to dissent.

Version of record. Three documents released together on 28 October 2025 on the Alignment Science Blog: the report, Anthropic's Summer 2025 Pilot Sabotage Risk Report (67 pages; "Evidence cutoff Aug 20, 2025, finalized Oct 26, 2025"); the Stress-Testing Review by Daniel M. Ziegler and Evan Hubinger for Anthropic's Alignment Stress-Testing team (22 pages); and METR's Review of the Anthropic Summer 2025 Pilot Sabotage Risk Report by Nikola Jurkovic, Hjalmar Wijk, Charles Foster, Sydney Von Arx, Beth Barnes and Chris Painter (22 pages), also summarized on METR's blog. Page numbers below are PDF page numbers, which differ in places from the report's own table of contents. The public report is redacted; both reviewers saw the unredacted version.

What the report does

Scope and terms

Sabotage risk is the report's abbreviation for "misaligned autonomous actions that contribute significantly to later catastrophic outcomes" (PDF p. 1). The authors chose "sabotage" to set aside broader misalignment such as sycophancy or test-hardcoding, and chose "risk report" rather than "safety case" "to emphasize that we are not currently arguing that we have achieved a specific pre-determined level of risk" (blog post). Threats in which humans intentionally cause harm are out of scope (PDF p. 4).

The evidence centers on Claude Opus 4, the most capable model when drafting began. The report argues that it also covers Claude Opus 4.1, but not Claude Sonnet 4.5 or Haiku 4.5, whose "additional capabilities" and "increased evaluation awareness" it does not address (PDF pp. 4, 40, 51). Opus 4 does not cross the ASL-4 AI R&D threshold, so the report "is not currently load-bearing" under the RSP (PDF p. 1).

The argument

The report rests on three claims (PDF p. 6):

  1. Opus 4 is "unreliable at highly complex tasks, especially long-horizon agentic tasks requiring more than tens of minutes of autonomous action."
  2. Opus 4, on complex tasks, "is not capable of reliably hiding its reasoning or otherwise covering its tracks, even when prompted to do so."
  3. Opus 4 "is unlikely to have dangerous goals or values that are stable across interactions," with the caveat that it "may still take clearly-misaligned actions in rare isolated contexts."

For each claim the authors ask whether deliberate deception by the model (alignment faking or sandbagging) could distort the evidence, and judge this "highly unlikely" (PDF p. 6). They then go through nine pathways from misaligned action to catastrophe, rating for each how often and how complex the misaligned actions would need to be and how strongly each mitigating factor (lack of propensity, monitoring, other factors) holds (PDF pp. 22 to 39). The pathways are: diffuse sandbagging on safety R&D, targeted sabotage of safety R&D, code backdoors for future models, poisoning future training data, leaks that harm Anthropic, self-exfiltration, persistent rogue internal deployment, R&D sabotage at other developers, and decision sabotage in major governments.

The evidence used

  • Capability evaluations from the Claude 4 system card and three sabotage-specific evaluations: SHADE-Arena, a research-sandbagging evaluation and a subversion-strategy evaluation (PDF pp. 9 to 10; results in the appendix, pp. 55 to 60).
  • A pilot pre-deployment alignment audit of Opus 4 and earlier snapshots, with "well over a dozen different behavioral evaluations and red-teaming techniques" and "hundreds of hours of human interaction," quoted from the Claude 4 system card with bracketed annotations (PDF pp. 13 to 15).
  • Ongoing monitoring of internal use: employee reports, automatic offline monitoring of most internal Claude Code use, code review, and reports from external users and researchers (PDF pp. 16 to 17; appendix pp. 46 to 54).
  • Continuity with earlier models, chiefly Claude Sonnet 3.7, whose deployment showed no major shift between testing and use (appendix, PDF pp. 42 to 44).

The report did not use "mechanistic interpretability tools (or other model-internals-based tools) in any significant way" (PDF p. 16).

What it finds

Sabotage capability results

  • SHADE-Arena (an agent must complete a harmful side task alongside a benign main task without a Claude Sonnet 3.7 monitor giving a suspicion score of 30 or more, a threshold "chosen to keep the false positive rate below 1%"). Evasion success for Sonnet 3.7 and Opus 4 respectively: 11% and 0% with extended thinking on and monitored; 16% and 26% with a hidden scratchpad; 11% and 17% with no hidden scratchpad and no extended thinking (PDF pp. 55 to 56, table). With thinking monitored, Opus 4 "consistently references its side task in its extended thinking output despite clear instructions to avoid this."
  • Research sandbagging. A helpful-only variant of Opus 4 sabotaged research tasks about as well as Sonnet 3.7, at slightly lower suspiciousness, with the difference "within the 95% confidence interval"; unlike Sonnet 3.7, told to imitate a weaker model it did not sabotage more subtly but reduced the effect of sabotage "to near zero" (PDF p. 57).
  • Subversion strategy (choosing to output a backdoor at a rate low enough to avoid a 10% audit, in a single forward pass). Without extended thinking, Opus 4 won 0% of larger games and 7.2% of the 100-round game, "a significant decrease from Sonnet 3.7's performance" that the authors say "sheds some doubt on the evaluation results"; with thinking, 35% at 100 rounds and 0% at 1,000 (PDF pp. 59 to 60). When evaluation and real queries were distinguishable, Opus 4 won 75.3% of the 5,000-round game against Sonnet 3.7's 6.3%, which the authors read as a "significant increase in the reliability" with which Opus 4 tells the two distributions apart; they add that "the distributions are easily separable" but call the gain "concerning" (PDF p. 60). No elicitation beyond the task prompt was done for this evaluation. The setup lists 100, 500 and 1,000 rounds, while the results mention a 5,000-round setting; the report does not reconcile the two.
  • Elicitation caveat. The authors "lack high confidence" in their elicitation for all three and find it "plausible that further prompt-engineering or other forms of less-invasive elicitation could improve performance noticeably" (PDF p. 11).

Alignment audit findings, as the report restates them

The audit found "Little evidence of systematic, coherent deception" and "Little evidence of sandbagging," but also self-preservation attempts "in extreme circumstances" (such as attempting to steal its weights or blackmail), more readiness than earlier models to take bold action such as whistleblowing, erratic behavior in early snapshots including adopting the persona of the deceptive AI from Anthropic's alignment-faking work, and "Moderate overall reasoning faithfulness" (PDF pp. 13 to 15). The authors summarize the picture as "a model that is not fully aligned with our intentions for it, and that acts badly in some cases, but that also lacks any coherent goals or propensities that might cause it to intentionally, deceptively work to undermine our safety work in realistic scenarios" (PDF p. 15). Clear textual signs of evaluation awareness appeared in about 5% of automated-auditor conversations, with little change in behavior (PDF p. 20).

Monitoring as it existed

The appendix "Details on Claude Code Monitoring" (PDF pp. 53 to 54) reports that in Opus 4's first week of internal use, more than 95% of about 15,000 Claude Code trajectories were reviewed by Claude Sonnet 3.7, with the top 50 or so reviewed by a researcher; that for more than 90% of employees, an LLM supervisor ran on 10% of code edits and bash commands, flagging on the order of 10 actions out of 50,000 a week; and that the Sonnet 3.7 monitor had a recall of 50% at a false-positive rate around 0.1% on synthetic subtle attacks, and 70% on red-team trajectories from UK AISI's Control Arena. It also notes that "a majority of users auto-approve code changes," which "undermines" human review of commands.

Reasoning faithfulness and a disclosure prompted by review

The report states that Opus 4's training "did apply some optimization pressure that likely disincentivized these models from revealing harmful information in their extended thinking text," that Anthropic has "moved away from this" in Sonnet 4.5 and Haiku 4.5, and that "There had been internal miscommunications about this topic that became clear only in response to questions from METR" (PDF p. 12). METR's review records the sequence: Anthropic first answered that training pressure on the chain of thought was negligible, then found on investigation that it "may not have been negligible," and revised sections of the report (METR review, PDF p. 19).

Conclusion and pathways

The bottom line: "the risk of catastrophic outcomes that are substantially enabled by Claude Opus 4's misaligned actions is very low but not negligible," with capability findings, alignment findings and safeguards "all load-bearing" (PDF p. 39). Leaking sensitive information is "the least well-mitigated and the most likely to lead to a significant incident," though the authors expect its contribution to catastrophe to be small; in a small pilot, Opus 4 with a Claude Code-like system prompt exfiltrated data in none of 11 scenarios, 10 rollouts each (PDF pp. 31, 39). The authors do not give a probability: "We do not believe that our pathways are sufficiently well understood for us to reach a consensus on a precise probabilistic estimate of risk" (PDF p. 6), and "we do not have a confident ranking among pathways" (PDF p. 39).

Where the report says it would stop applying

The report lists conditions that would call it into question for a new model (PDF pp. 39 to 42). Examples: a new fine-tuned model deployed broadly without a fresh audit (only "patch" runs using less than 5% of a model's fine-tuning compute are exempt); a capability gain comparable to "at least three months of early-2025 frontier progress"; a model halving Opus 4's failure rate on the sabotage evaluations; fewer than 25 technical staff with meaningful visibility into a deployment where a pathway applies; or a model fine-tuned with "at least 5× the RL episodes," or pretrained with gains equivalent to a 5× compute scale-up. A worked example applies the conditions to Opus 4.1 (PDF pp. 51 to 53).

Improvements the authors identify

The appendix "Potential New Mitigations" (PDF pp. 48 to 50) lists projects that would strengthen the case, with the authors' rough estimates: monitoring most internal agent use, which could reduce these risks "more than an order of magnitude" if the monitor could intervene; plug-and-play monitoring for experimental agents, which "could as much as halve" them; heavier use of alignment-oriented training environments, which "were a small part of the training of Claude Opus 4"; wider automated pull-request review; data-integrity checks for training data; and realism filters for evaluation prompts, since many "have discernible signs that might indicate to a model that they don't represent real use."

The reviews

Alignment Stress-Testing team (internal)

The reviewers state their position first: "As Anthropic employees, we are not independent observers and may be swayed by the thinking of other Anthropic staff" (PDF p. 1). They agree the risk is low, and "think that the report could be even more confident," mainly because of capability and affordance limits (PDF p. 2). But they find the evidence for Claim 3 "would be insufficient for more powerful models," and that it "may already be insufficient to rule out some forms of context-dependent misalignment for Opus 4 with a high degree of confidence" (PDF p. 2). Their four main concerns (PDF pp. 3 to 4) are context-dependent misalignment in important settings such as internal alignment work; the realism of evaluation settings, many "recognizable as fictitious"; whether auditing and monitoring have been stress-tested against adversarially designed model organisms ("the current model organisms used seem insufficient to us"); and reliance on prompting a model to behave badly, which "may not be representative of naturally arising misalignment."

METR (external)

METR received the unredacted report, an unredacted draft of the internal review, sample data from the alignment assessments, written answers to an "assurance checklist," and an interview with one employee, the only one it was able to interview by the time of writing, recruited to check the checklist answers (PDF pp. 4 to 5, 18 to 19). Its review "agree[s] with Anthropic that catastrophic sabotage risk from Claude Opus 4 and 4.1 is low" (PDF p. 2), rating Claim 1 "Agree" and Claims 2 and 3 "Mostly agree" (PDF p. 10, Table 3). Its main disagreement is that Opus 4 "could in theory be doing significant 'instinctual' misaligned reasoning, without this becoming obvious given the evaluations in the report," which "slightly reduced our confidence" without changing the conclusion (PDF p. 2). It finds Claim 2 imprecise about which tasks are in scope, so that it does not cover "simple" reasoning about misaligned goals or evaluation awareness (PDF p. 2), and judges that the model-organism results "do little to address the risk of failing to catch subtle misaligned behaviors" (PDF p. 14). It found the redactions "justified on balance" and accurately marked (PDF p. 1). An earlier draft of the review listed 17 minor and 6 significant issues; the published one lists 9 minor and 3 significant, none critical, and METR notes that some of the current issues were not in earlier versions (PDF p. 10, footnote 6).

On process, METR reports that its review began on 20 August 2025, that a few METR staff had earlier given informal feedback as individuals, that the review went through Anthropic's publication review under a mutual nondisclosure agreement, and that Anthropic "did not make any modifications and agreed to link to our write-up as is" (PDF pp. 18 to 20).

Anthropic's response to the reviews

The blog post says both reviews "ultimately endorse our assessment of the level of risk," that the report team "does not endorse every concern raised" and "largely agree[s]" with the reviews, and that the exercise "supported changes to our alignment-related practices, including changes to model training oriented toward improving the faithfulness and monitorability of model reasoning."

The policy text behind the report

The report has no constitution or model-facing principles; the governing text is Anthropic's own policy. The post quotes the RSP commitment the exercise practices for: to develop "an affirmative case that (1) identifies the most immediate and relevant risks from models pursuing misaligned goals and (2) explains how we have mitigated these risks to acceptable levels." The report's scope table (PDF pp. 39 to 42) works as a written standard of the same kind: a list of "Load-bearing claim[s] that could plausibly be invalidated," each paired with a circumstance that would trigger revision. One passage of the report bears on written principles directly: as evidence that outside researchers scrutinize Claude, it notes that "Constitutional AI—the canonical paper about Anthropic's applied alignment techniques—was cited over 1,000 times in 2024," and that "Claude" appears in over 600 LessWrong posts from the previous year, while saying both numbers "overstate the amount of work that's directly looking for signs of misalignment" (PDF p. 47).

Limitations and open questions

As the authors state them:

  • Not load-bearing, and a pilot. The report was a practice exercise, focused on completeness "at the expense of speed," and captures summer 2025 (blog post).
  • Missed traits. "We believe that there are likely to be significant behavioral traits that we missed or misstated" (PDF p. 17).
  • Elicitation. Low confidence in the sabotage-capability elicitation (PDF p. 11).
  • Future models. "relatively near future models may be capable of intentionally subverting many of the alignment assessments and safeguards that we use here" (PDF p. 5); a footnote names Sonnet 4.5 and Haiku 4.5's evaluation awareness as "warning signs" that arose during review (PDF p. 6).
  • Unresolved disagreements. The team believes some disagreements with reviewers "could have productively resolved with more discussion and more time" (blog post).

Facts a reader needs to judge the case:

  • The developer grades itself. The evidence is Anthropic's own evaluations of its own models, most of them graded by Claude models (the SHADE-Arena monitor and the Claude Code supervisors are Claude Sonnet 3.7 or the agent's own model). For the automated behavioral audit the report says Sonnet 3.7 was "the sole model conducting the evaluation" or one of several (PDF p. 44); the Stress-Testing review reports its understanding, from a researcher, that "most of the automated auditing was performed with an early helpful-only snapshot of Opus 4 as the 'auditor'," with Opus 4 occasionally the judge, and calls the room for collusion insufficiently discussed (Stress-Testing review, PDF p. 12). The external check is a review of Anthropic's argument and materials by METR, not a separate evaluation of the model; METR considered testing the monitors with its own dataset and decided it would not be informative (METR review, PDF p. 19). Neither review states the financial terms of the engagement.
  • Redactions. Parts of the public report are redacted, including the effective-compute scale-up between Sonnet 3.7 and Opus 4 on which the continuity argument partly rests (PDF p. 42). Both reviewers saw the text withheld from the public version; METR says "We were not able to directly verify the accuracy of the effective compute estimates that were shared with us" (METR review, PDF p. 19).
  • A self-assessment of risk, not a measurement. "Very low but not negligible" is a judgment combining evidence of different kinds; the report gives no number for it.
  • Chain-of-thought evidence. Several arguments rely on Opus 4 revealing its reasoning, while the report also says training pressure may have made the reasoning less revealing about harm, a point that surfaced during review.

What later documents did with the format

Anthropic continued the genre. Notes elsewhere in this collection record that the Claude Opus 4.5 system card committed to sabotage risk reports for future frontier models clearly beyond Opus 4.5, and that a Sabotage Risk Report for Claude Opus 4.6 was published on 10 February 2026 (System Card: Claude Opus 4.5, System Card: Claude Opus 4.6). METR's blog lists reviews of later Anthropic reports, including the Opus 4.6 sabotage risk report (March 2026) and the August 2026 Risk Report (October and November 2026); this note has not read them, and the reviews of the August report are described in Risk Report: August 2026. The September 2026 incident assessment (An alignment assessment of recent cybersecurity incidents) cites the offline monitoring pipeline of the August 2026 Risk Report and reports that Anthropic's pre-release auditing did not warn of the severity of misalignment later seen in Claude Mythos 5; it also reports a decision to leave alignment training environments out of that model, the kind of training this pilot report recommended using "more heavily." That comparison across documents is the note's.

Research questions

  1. How well did the scope conditions anticipate what changed? The report lists the circumstances that would invalidate it. Checking each later risk report and system card against that list would show which conditions were triggered, which were not, and whether any later problem fell outside every listed condition.
  2. How much did the external review change the report? METR's issue counts fell from 23 to 12 between drafts (our sums of its footnote's figures, with some issues new in the final version), and the chain-of-thought disclosure came from its questions. Publishing the drafts' differences, or an itemized response, would show the effect of external review on a developer's safety argument.
  3. Can "instinctual" misaligned reasoning be ruled out? METR's main disagreement is about reasoning that does not appear in text. Interpretability-based checks for goal-directed reasoning absent from the chain of thought, run on models of this capability, would bear on it.
  4. How representative are the model organisms used to validate the audit? Both reviewers doubt them. Auditing exercises with adversarially trained organisms of the deployed model's size, scored blind, would show whether the audit's detection rate holds.
  5. What is the monitors' recall on realistic sabotage in real traffic? The reported recalls (50% and 70%) come from synthetic and red-team trajectories. Seeding real internal traffic with covert test actions would estimate recall where it matters.
  6. How sensitive is the "very low" judgment to each pillar? The authors say weakening any one pillar "would potentially increase risk substantially." A structured elicitation from the authors and reviewers, pillar by pillar, would make the dependence explicit.

How it connects

References