On 22 October 2024 Anthropic released an upgraded version of Claude 3.5 Sonnet and announced Claude 3.5 Haiku, and added to the Claude 3 model card "an addendum covering an upgrade to Claude 3.5 Sonnet and Claude 3.5 Haiku" (Claude 3 card, p. 1). The addendum is also published as a standalone 14-page PDF titled "Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet". The addendum's tables label the upgraded model "Claude 3.5 Sonnet (New)", and later Anthropic documents call it "Claude 3.5 Sonnet (new)" (for example the Claude 3.7 Sonnet System Card, Section 1.1). The headline addition is computer use: the upgraded Sonnet can "interpret screenshots of a Graphical User Interface (GUI) and generate appropriate tool calls to perform requested tasks" (Section 1, p. 1). The addendum describes its own purpose as to "document the progress we've made" and to give "our users and the broader AI community" information about the new models, "both their enhanced capabilities and our ongoing commitment to safety and ethical AI development" (Section 1, p. 1).
Structure of the card
The addendum has three sections and an appendix:
- Introduction (about a page), with a subsection on knowledge cutoff.
- Evaluations (about six pages, pp. 1 to 7): computer use (OSWorld), tool use and agentic tasks (SWE-bench Verified, TAU-bench, the internal agentic coding evaluation), vision, refusals, reasoning and coding benchmarks, and human feedback.
- Safety (about three pages, pp. 7 to 10): Trust and Safety red teaming, prompt injection, computer use red teaming, an RSP review of computer use, frontier risk evaluations, and a closing commitment.
- Appendix A: worked examples of computer use.
Compared with the June 2024 addendum (eight pages, no Trust and Safety section), this one adds safety material specific to a new capability (computer use and prompt injection) and brings back a Trust and Safety section, now with red teaming in six languages. It still contains no training section beyond knowledge cutoffs and no limitations section. The next document in the series, the Claude 3.7 Sonnet System Card (February 2025), is standalone and organized around harms rather than benchmarks.
Training and the written principles
Knowledge cutoff. The upgraded Claude 3.5 Sonnet has an April 2024 cutoff, the same as the original Claude 3.5 Sonnet; Claude 3.5 Haiku's cutoff is July 2024 (Section 1.1, p. 1).
What is not stated. The addendum says nothing about pretraining data, RLHF, Constitutional AI, the constitution or character. The Claude 3.7 Sonnet card later states that the disability-rights principle from Collective Constitutional AI was added "Starting with Claude 3.5 Sonnet (new)", that is, with the upgraded model this addendum covers (3.7 card, Section 1.1, p. 3); this addendum does not mention it, and the Claude 3 card places the same addition at Claude 3 (Claude 3 card, pp. 3 to 4).
Training on adversarial prompts and the system prompt. One training change is described: both models were trained to "recognize and resist prompt injection attempts", using internal test sets of attacks and training "on adversarial interactions" (Section 3.1.2, p. 8). A prompt injection, in the addendum's definition, is an attack in which "a malicious user feeds instructions to a model that attempt to change its originally intended behavior"; the models are said to be "better able to recognize adversarial prompts from a user and behave in alignment with the system prompt" (p. 8). This is the first card in the series to describe training a model to keep to its system prompt against conflicting instructions.
Guidance to deployers. For computer use the addendum recommends "using a dedicated virtual machine, limiting access to sensitive data, restricting internet access to required domains, and keeping a human in the loop for sensitive tasks" (p. 8). The policy referred to throughout is now called the "Usage Policy" (pp. 1, 3 and 9).
Alignment, honesty and character assessments
As in the earlier addendum, there is no alignment assessment. Behavior is reported through refusals, human preference and Trust and Safety red teaming.
Refusals. On the same WildChat and XSTest evaluations used since the Claude 3 card, Table 6 (p. 5) reports:
| Measure | 3.5 Sonnet (new) | 3.5 Haiku | 3.5 Sonnet | 3 Opus |
|---|---|---|---|---|
| WildChat toxic, correct refusals (higher is better) | 89.2% | 88.0% | 96.4% | 92.0% |
| WildChat non-toxic, incorrect refusals (lower is better) | 5.3% | 4.8% | 11.0% | 11.9% |
| XSTest, incorrect refusals (lower is better) | 4.3% | 2.0% | 1.7% | 8.3% |
Both new models refused fewer harmless WildChat prompts than the original Claude 3.5 Sonnet, and also refused fewer toxic ones (89.2% for the upgraded Sonnet and 88.0% for Haiku, against 96.4% for the original). On XSTest the upgraded Sonnet's incorrect refusal rate is higher than the original's. The addendum presents the table without discussing these changes (Section 2.4, p. 5).
Human preference. Against the original Claude 3.5 Sonnet as the 50% baseline, the upgraded model's win rates were 61% for document analysis, 57% for visual understanding, 58% for creative writing, 52% for coding and 51% for following precise instructions (Section 2.6, p. 6). Figure 2 (p. 8) includes adversarial "Honesty" and "Harmlessness" panels, not discussed in the text.
Trust and Safety red teaming. Testing covered "fourteen policy areas in six languages: English, Arabic, Spanish, Hindi, Tagalog, and Chinese", with attention to elections, child safety, cyber attacks, hate and discrimination, and violent extremism (Section 3.1.1, p. 7). Overall harm rates for the upgraded Sonnet were "similar to, but slightly improved over" the original; Claude 3.5 Haiku improved on Claude 3 Haiku, particularly for non-English prompts. Both models "struggled with nuanced requests or those framed as fiction, roleplaying, or artistic content" (p. 7). No rates are printed.
Safety evaluations and deployment decisions
Computer use red teaming. Trust and Safety identified potential abuse vectors, "though none were deemed imminent": scaled account creation, scaled content distribution, age assurance bypass and abusive form filling (Section 3.1.3, pp. 8 to 9). Mitigations include monitoring protocols and new classifiers for computer use (p. 9).
Computer use and the RSP. Anthropic asked whether computer use changes its frontier-risk threat models and concluded that current threat models and evaluations "adequately capture the risks of computer use" at this capability level (Section 3.1.4, p. 9). The reasoning is given per domain: computer use alone is unlikely to raise CBRN risk without the underlying knowledge; it may lower the barrier for novices in cyber but those actors "likely lack the additional knowledge to present an extreme threat"; and visual computer use is not "on the critical path" to the software engineering skills that the autonomy evaluations test (p. 9).
Frontier risk evaluations. For the upgraded Sonnet, Anthropic evaluated CBRN (automated knowledge tests, non-expert uplift, new baselines and manual red teaming), cybersecurity (capture-the-flag challenges in pwn, reverse engineering, cryptography, web and network security) and autonomy (software engineering tasks such as submitting a pull request that passes tests) (Section 3.2, p. 9). The US AI Safety Institute and UK AI Safety Institute conducted joint pre-deployment testing of the upgraded Sonnet, and METR conducted an independent assessment (pp. 1 and 9).
Result. The upgraded Sonnet "showed increased capabilities in risk-relevant areas" and "stronger capabilities across all domains", but "did not demonstrate capabilities requiring ASL-3 safeguards and security in any domain"; Anthropic judged that it "does not require ASL-3 safeguards at this time" (Section 3.2.1, pp. 9 to 10). The addendum gives no scores or thresholds for these evaluations. It states that "Both models underwent extensive safety evaluations" in the biological, cybersecurity and autonomy domains (pp. 1 and 9), but the results in Section 3.2.1 and the ASL judgment are given only for the upgraded Sonnet, not for Claude 3.5 Haiku.
Scope of the evaluations. Section 3.2 calls these "comprehensive safety evaluations" (p. 9), and Section 3.3 says the models "did not trigger the full evaluation protocol described in our Responsible Scaling Policy" (p. 10). The June addendum gave the reason for the same situation with the original Claude 3.5 Sonnet: the model did not reach the RSP's effective-compute threshold for the full protocol. The October addendum does not state why the full protocol was not triggered for these models.
Capabilities reported
Because computer use is the new capability, the main numbers are recorded here. On OSWorld, a benchmark of real computer tasks in web and desktop applications, the upgraded Sonnet succeeded on 14.9% of tasks using screenshots only and 15 steps, and on 22% when allowed 50 steps with an optimized prompt; the human success rate cited is 72.36% (Section 2.1, p. 2, Table 1). On SWE-bench Verified, which resolves real GitHub issues and grades with hidden unit tests, the upgraded Sonnet scored 49.0% and Claude 3.5 Haiku 40.6%, against 33.4% for the original Claude 3.5 Sonnet (Section 2.2, p. 3, Table 2). On TAU-bench, simulated customer-service tasks with policies to follow, the upgraded Sonnet solved 69.2% of retail and 46.0% of airline cases (p. 3, Table 3). On the internal agentic coding evaluation from the June addendum, the upgraded Sonnet solved 78% and Haiku 74%, against 64% for the original Claude 3.5 Sonnet (p. 4, Table 4). Claude 3.5 Haiku launched as text-only, so no vision results are given for it (p. 5). A footnote explains that comparisons exclude OpenAI's o1 models because they "depend on extensive pre-response computation time" (footnote 1, p. 2).
What the card says it does not know
Stated limits are few and specific to computer use: performance "remains well below human performance" on OSWorld (p. 2); the feature is "still in its early stages of development" (p. 3); and Claude's reliability on computer tasks "is not yet on par with human performance" (p. 9). On safety, the addendum commits to further research "As computer use capabilities evolve" (p. 9) and to continued testing (Section 3.3, p. 10). It has no general limitations section.
Research questions
- Did prompt injection training change how the model weighs system prompts against user instructions in ordinary use? The addendum trains the models to "behave in alignment with the system prompt" under attack but reports no injection success rates. Running the 176-task injection evaluation that the Claude 3.7 Sonnet card later reports (for Claude 3.7 Sonnet only) on this model would give a baseline; comparing ordinary instruction-following with and without that training would show side effects.
- Why did correct refusals of toxic WildChat prompts fall? The upgraded Sonnet refused 89.2% of toxic prompts against 96.4% for the original. Category-level data, and labels distinguishing harmful compliance from acceptable engagement (the distinction the Claude 3.7 Sonnet card later formalizes as "appropriate harmlessness"), would show whether this is more harm or fewer unnecessary refusals.
- What did the frontier-risk evaluations find for Claude 3.5 Haiku? The addendum says both models were evaluated but reports results and an ASL judgment only for the upgraded Sonnet. Haiku's results on the same CBRN, cyber and autonomy evaluations, or a stated capability comparison used in place of them, would answer it.
- When did the disability-rights principle enter training? The Claude 3.7 Sonnet card dates the principle to the model this addendum covers, yet the addendum does not mention it. A version history of the constitution would resolve the disagreement with the Claude 3 card.
How it connects
The addendum refers to the Claude 3 model card for the refusal evaluations (Section 5.4.1) and win-rate method (Section 5.5), and to the Claude 3.5 Sonnet Model Card Addendum for the internal agentic coding evaluation (p. 4). It cites no Anthropic research papers. The computer-use capability it documents is described in the planned note Developing a computer use model, which the addendum does not cite. The next card in the series is the Claude 3.7 Sonnet System Card, which reports computer-use prompt injection results and compares Claude 3.7 Sonnet with this model throughout.
References
- Anthropic (2024). Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet. October 2024, 14 pp. https://www-cdn.anthropic.com/c7822cdc35ad788ec87e14b3a9d45010f1f86c38.pdf. Also appended to the Claude 3 model card PDF: https://www-cdn.anthropic.com/c6a80a657af445f40e31afac050f3bf76d3b1404.pdf
- Anthropic (2024). Claude 3.5 Sonnet Model Card Addendum. June 2024 (for the effective-compute threshold).
- Anthropic (2025). Claude 3.7 Sonnet System Card. February 2025 (for the name "Claude 3.5 Sonnet (new)" and the dating of the disability-rights principle).
- Xie, T., et al. (2024). OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments (cited by the addendum for OSWorld and the human baseline; not read for this note).