Claude 3.5 Sonnet Model Card Addendum

Claude 3.5 Sonnet was released on 20 June 2024, the date on which the Claude 3 model card records that "an addendum covering Claude 3.5 Sonnet" was added (Claude 3 card, p. 1). The addendum is an eight-page PDF, and it explains its own form in its first paragraph: "Since it is an evolution of the Claude 3 model family, we are providing an addendum rather than a new model card" (Section 1, p. 1). It covers one model, which it describes as outperforming Claude 3 Opus "while operating faster and at a lower cost" (p. 1), and it provides "updated key evaluations and results from our safety testing" (p. 1). Everything not updated, including training data, training process, intended and prohibited uses, and the constitution, is left to the main card by implication; the addendum does not restate or refer to those sections.

Structure of the card

The addendum has three sections and a reference list (pp. 7 to 8):

1. Introduction (one paragraph).
2. Evaluations (about four pages, pp. 1 to 5): reasoning, coding and question-answering benchmarks (Table 1), vision benchmarks (Table 2), an internal agentic coding evaluation (Table 3), refusals (Table 4), needle-in-a-haystack recall (Table 5), and human preference win rates (Section 2.1, Figure 3).
3. Safety (one page, p. 6): why the full RSP protocol was not required, what was evaluated anyway, and the ASL classification.

There are no sections on training, model details, Trust and Safety red teaming, societal impacts or areas for improvement. Against the Claude 3 card's 42 pages, this is the most compressed document in the series. The October 2024 addendum for Claude 3.5 Haiku and the upgraded Claude 3.5 Sonnet keeps the addendum form but runs to 14 pages and restores a Trust and Safety section; in February 2025 the Claude 3.7 Sonnet System Card returns to a standalone document under a new name.

Training and the written principles

The addendum says nothing about how Claude 3.5 Sonnet was trained: no data cutoff, no mention of RLHF, Constitutional AI, character or the constitution, and no system prompt or usage policy text. Its only description of the model is comparative ("an evolution of the Claude 3 model family", "improved capabilities, including better coding and visual processing", p. 1). Later documents fill in two facts. The October 2024 addendum gives the original Claude 3.5 Sonnet's knowledge cutoff as April 2024 (that addendum, Section 1.1, p. 1). The Claude 3.7 Sonnet card says the disability-rights principle drawn from Collective Constitutional AI was added "Starting with Claude 3.5 Sonnet (new)" (that card, Section 1.1, p. 3), which would place it after this model; the Claude 3 card places it at Claude 3 (Claude 3 card, Sections 2.6 and 4.1, pp. 3 to 4).

Alignment, honesty and character assessments

The addendum reports no alignment assessment in the later sense. The evaluations that bear on behavior are refusals and human preference.

Refusals. Using the WildChat and XSTest datasets described in the main card, the addendum reports that Claude 3.5 Sonnet "has fewer incorrect refusals and more correct refusals" than Claude 3 Opus (Refusals, p. 3). Table 4 (p. 3) gives:

Two details complicate the summary sentence. On WildChat non-toxic prompts, Claude 3.5 Sonnet refused more often than Claude 3 Sonnet and Haiku, though less often than Opus. And the table gives Claude 3 Opus an XSTest rate of 8.3%, where the main card's text gives "9%" for the same model (Claude 3 card, Section 5.4.1, p. 10); the addendum does not comment on the difference.

Human preference. Raters compared models under task-specific instructions, with Claude 3 Opus as the 50% baseline (Section 2.1, pp. 3 and 5). The text reports win rates for domain experts "as high as 82% in Law, 73% in Finance, and 73% in Philosophy" (p. 3). Figure 3 also includes adversarial "Honesty" and "Harmlessness" panels; the addendum does not discuss them in the text.

Safety evaluations and deployment decisions

Why the full protocol was not run. The RSP then in force set an effective-compute threshold at which its full evaluation protocol is run. The addendum states that Claude 3.5 Sonnet, although more capable than Opus, "does not trigger the 4x effective compute threshold at which we will run the full evaluation protocol" (Section 3.1, p. 6). It refers readers to a separate "Responsible Scaling Policy Evaluations Report" for Claude 3 Opus (May 2024, reference [23]).

What was done anyway. The addendum says Anthropic considers regular testing important "even if not formally required by our RSP", and ties this to voluntary commitments made to the White House, the G7 and the Seoul AI summit (p. 6). Safety teams evaluated CBRN (chemical, biological, radiological and nuclear) risks, cybersecurity and autonomous capabilities (p. 6):

• CBRN: automated knowledge tests and measurement of how much the model improved non-expert performance.
• Cyber: capture-the-flag challenges on vulnerability discovery and exploit development.
• Autonomy: writing code that passes pre-defined tests, similar to the internal agentic coding evaluation.

For each domain Anthropic defined quantitative "thresholds of concern" as a conservative early sign of approaching ASL-3; had any been crossed, a council including the Responsible Scaling Officer, evaluation leads and external experts would have decided on further evaluation or stronger protections (Section 3.2, p. 6).

Refusals during capability testing. Because harmlessness training can make a model refuse during capability tests and so understate what it can do, Anthropic measured refusal rates by topic and used "internal research techniques to acquire non-refusal model responses" to estimate how a helpful-only version would have performed (p. 6). The addendum adds that these estimates may still understate an equivalent helpful-only model, "because alignment training may cause it to hedge concerning answers" (p. 6).

External testing. The UK AI Safety Institute tested a near-final model before deployment and shared results with the US AI Safety Institute; METR "did an initial exploration of the model's autonomy-relevant capabilities" (p. 6).

Result. Anthropic "observed an increase in capabilities in risk-relevant areas compared to Claude 3 Opus", consistent with incremental training and elicitation, and the model did not exceed the thresholds; it is classified ASL-2 (Section 3.3, p. 6). No evaluation scores or threshold values are given for any domain.

Capabilities reported

The addendum leads with capabilities, so a brief record: Claude 3.5 Sonnet scored 59.4% on GPQA Diamond (0-shot chain of thought), 90.4% on MMLU (5-shot chain of thought) and 92.0% on HumanEval (Table 1, p. 2). On an internal agentic coding evaluation, in which the model must implement a real pull request in an open-source codebase so that hidden tests pass, it solved 64% of problems against 38% for Claude 3 Opus, 21% for Claude 3 Sonnet and 17% for Claude 3 Haiku (Section 2, p. 1; Table 3, p. 3). Average needle-in-a-haystack recall was 99.7% at all context lengths and at 200K tokens (Table 5, p. 3).

What the card says it does not know

The addendum's only stated uncertainties are in the safety section: "We still have a lot to learn about the science of evaluations and ideal cadences for performing these tests" (Section 3.1, p. 6), and the possibility that capability estimates understate a helpful-only model (p. 6). It has no limitations section and does not repeat the main card's.

Research questions

1. What does an addendum carry over from the main card? The addendum does not say whether training data, the constitution, the AUP and the stated limitations of the Claude 3 card apply unchanged to Claude 3.5 Sonnet. A comparison of the Claude 3 card, this addendum and the October 2024 addendum, against Anthropic's statements about what changed between models, would show which claims a reader can safely carry forward.
2. How was "4x effective compute" measured for this model? The decision not to run the full protocol rests on that threshold, but the addendum gives no estimate of the model's effective compute relative to Opus. The RSP version of the time and any later disclosure of the estimate would answer it.
3. How large were the increases in risk-relevant capability? The addendum reports an increase without numbers. Publication of the domain scores against the "thresholds of concern", as later cards do for some evaluations, would show how much margin remained.
4. Why did correct and incorrect refusals both rise on WildChat relative to Claude 3 Sonnet? Claude 3.5 Sonnet refused more toxic and more non-toxic WildChat prompts than Claude 3 Sonnet (Table 4). Category-level refusal data would show whether this reflects a general shift toward refusing or specific prompt types.

How it connects

The addendum depends on the Claude 3 model card for everything it does not restate, and refers to it for how win rates are calculated (footnote 3, p. 3). Its successor in the series is the Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet (October 2024), which reruns the same internal agentic coding evaluation and refusal tables. The addendum cites no Anthropic research papers.

References

• Anthropic (2024). Claude 3.5 Sonnet Model Card Addendum. June 2024, 8 pp. https://www-cdn.anthropic.com/fed9cc193a14b84131812372d8d5857f8f304c52/Model_Card_Claude_3_Addendum.pdf. Also appended to the Claude 3 model card PDF: https://www-cdn.anthropic.com/c6a80a657af445f40e31afac050f3bf76d3b1404.pdf
• Anthropic (2024). Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet. October 2024 (for the original model's knowledge cutoff).
• Anthropic (2025). Claude 3.7 Sonnet System Card. February 2025 (for the dating of the disability-rights principle).
• Anthropic (2024). Responsible Scaling Policy Evaluations Report: Claude 3 Opus. May 2024 (cited by the addendum as reference [23]; not read for this note).