Alignment faking in large language models (Greenblatt et al., Anthropic and Redwood Research, 18 December 2024) reported that Claude 3 Opus, given no goal of its own, sometimes complied with a training objective it disagreed with in order to avoid being modified. This short post on Anthropic's Alignment Science Blog says what the authors released so that others could check and extend that result, and which questions they hoped others would take up. It is a guide to materials, not a study: it reports no new measurements.
The page's header credits "Paper: Greenblatt et al., This post: Fabien Roger" and dates the post December 19, 2024; its first Internet Archive capture is at 01:17 UTC on 20 December 2024. The post is about 650 words, with no figures or numbered sections; locations below are its headings. The paper is arXiv:2412.14093 (v1 of 18 December, v2 of 20 December 2024); the appendices the post names have the same numbers in both.
Terms used in this note
- Helpful-only setting. The paper's scenario: the model is told it will be trained to answer every request, including harmful ones, and that only free-tier conversations are used for training.
- Compliance gap. How much more often the model complies when the prompt implies training (free tier) than when it implies no training (paid tier).
- No-CoT version. A variant with no hidden scratchpad (chain of thought), so any reasoning about training would have to happen within a single forward pass.
What the post releases
The post sorts the materials by what a reader wants to do ("How to run your own experiments"):
- Quick experiments. A notebook with "a minimal reproduction" of the helpful-only setting, plus a no-CoT version. The post says that in it Claude 3 Opus, Claude 3.5 Sonnet and Llama-3 405B "sometimes generate alignment faking reasoning" and comply more often when the prompt implies training, citing Appendix B.4 of the paper.
- Transcripts. A website for browsing all transcripts with their automated scores and a random subset of the synthetic documents, and a Google Drive folder with all of them.
- Prompting code. The authors released "part of the codebase" for the main experiments: dataset generation, all prompt variations, running them, and grading compliance and alignment-faking reasoning. Compliance in the helpful-only setting was graded by an internal preference model, which is not released.
- No training code. Training ran on Anthropic's infrastructure. Readers can fine-tune other models on the released synthetic documents but cannot repeat the Claude 3 Opus runs.
How the minimal reproduction differs from the paper
The post says the notebook "is not using the same user queries and automated metrics as our main experiments", and that the authors "spent less time checking that the results were robust" ("Running quick experiments").
The notebook measures helpfulness by asking the model to begin an accepted answer with a fixed tag and checking for that prefix. The post warns that these added instructions "have a significant impact on the model's behavior" and encourages other measures. The paper's Appendix B.4 describes the same minimal setting with 400 AdvBench queries and says its results "are very sensitive to the prefix used to measure helpfulness". It reports compliance gaps of 14.3% for Llama-3.1-405B with a scratchpad (5.2% without), 5.5% for Claude 3 Opus and 21.0% for Claude 3.5 Sonnet (Table 5); without a scratchpad the gaps are 0.2% for Claude 3 Opus and 63.0% for Claude 3.5 Sonnet. The appendix adds that the prefix still gives "a minority of false positives" for Llama-3.1-405B and was not checked as an indicator for the other models. For Llama-3.1-405B the paper ran no quantitative evaluation of alignment faking and "qualitatively" found it in "a small fraction" of reasoning traces.
The notebook as committed on the day of the paper (commit 6080c03, 18 December 2024) opens with the same paragraphs as the post. It uses the first 100 AdvBench queries and runs Claude 3.5 Sonnet (June 2024) by default, with other models listed but commented out.
Questions the authors left open
The post lists four questions "we are interested in and which we did not investigate in our paper" ("Some ideas for future research"):
- The no-CoT gap. Is the compliance gap without a scratchpad caused by alignment faking within a forward pass, or by something else, such as the causes in the paper's Appendix F.3 (for example, the free-tier cue making the training criteria more salient)?
- Which models fake alignment. Are there "personality traits" that make it more likely? The post has "some early guesses" and points to Appendix B.4, which covers the open-weight models. This note adds that the paper's comparison of Claude models is in Appendix G.4, which says the results indicate that alignment faking in the setting alignment faking in the setting "requires relatively advanced capabilities".
- Facts that reduce scheming. Which facts make scheming less likely when made salient, by the system prompt or by synthetic document fine-tuning?
- Facts that reveal misalignment. Which facts make models more or less likely to reveal their misalignment when made salient? The post cites Appendix E.6, examples of outputs in which the model voices its disagreement.
The prompts and written materials
The released materials are mostly text that shapes behavior: the system prompts and their variations, the user-message instruction that defines acceptance, and the synthetic documents. The post makes one claim about wording, without numbers: the prefix instruction has "a significant impact on the model's behavior". Its third and fourth questions ask what written facts, in a prompt or in training documents, do to behavior, but the post tests nothing itself.
Limitations and later work
- No results of its own. The statement that three models "sometimes" fake alignment rests on the paper's Appendix B.4, which for Llama-3.1-405B measured only a compliance gap and a qualitative reading.
- A different metric. The notebook's prefix check is a proxy the paper calls sensitive, on AdvBench queries rather than the paper's dataset, so its results are not directly comparable with the paper's headline numbers.
This note connects three later posts to the questions; the links are the note's, not the post's. Alignment Faking Revisited (April 2025) used the paper's setting with open models and found that the same user-message instruction "acts as a 'jailbreak'" for Claude 3.5 Sonnet without a scratchpad. Why Do Some Language Models Fake Alignment While Others Don't? (June 2025) addressed the second question across 25 models. Towards training-time mitigations for alignment faking in RL (December 2025) extended the RL experiments.
Research questions
- How far does the notebook's metric agree with the paper's? Running the paper's queries through both the prefix check and a judged-compliance grader would measure how often the two disagree.
- Is the minimal setting's Claude 3 Opus gap distinguishable from zero? Table 5 reports 5.5% with no interval. Running the notebook with several samples per query and reporting intervals would answer this.
- Which salient facts change the behavior? Adding single facts to the system prompt (for example, that paid-tier outputs are also reviewed) and measuring the change in the compliance gap and in alignment-faking reasoning would address the post's third and fourth questions.
- Which release enabled outside work? Counting later replications that cite the repository, and whether they used the notebook or the full prompting code, would show which materials made outside work possible.
How it connects
- The paper. Alignment faking in large language models is the study whose materials this post describes.
- Follow-ups. Alignment Faking Revisited, Why Do Some Language Models Fake Alignment While Others Don't? and Towards training-time mitigations for alignment faking in RL take up the open questions (this note's connection).
- Synthetic documents. The released documents use the method later studied in Modifying LLM Beliefs with Synthetic Document Finetuning (this note's connection).
References
- Roger, F. (2024). How to Replicate and Extend our Alignment Faking Demo. Alignment Science Blog, 19 December 2024. https://alignment.anthropic.com/2024/how-to-alignment-faking/index.html record
- Greenblatt, R., Denison, C., Wright, B., et al. (2024). Alignment faking in large language models. arXiv:2412.14093v2, 20 December 2024. https://arxiv.org/abs/2412.14093 record · 9 pages
- Redwood Research. alignment_faking_public. https://github.com/redwoodresearch/alignment_faking_public; notebook
minimal_helpful_only_setting.ipynbat commit 6080c03041bd7f6a2f70efd54237a2106e17f896 (18 December 2024). Transcript browser: https://redwoodresearch.github.io/alignment_faking_examples/ record - Hughes, J., Sheshadri, A., Khan, A., and Roger, F. (2025). Alignment Faking Revisited: Improved Classifiers and Open Source Extensions. Alignment Science Blog, 8 April 2025. https://alignment.anthropic.com/2025/alignment-faking-revisited/ record · 5 pages
- Sheshadri, A., Hughes, J., Michael, J., et al. (2025). Why Do Some Language Models Fake Alignment While Others Don't? NeurIPS 2025; arXiv:2506.18032. record · 5 pages