Can AI agents do alignment research that measurably works, and can that be checked? This study picks the part of alignment research where checking is easiest: fixing alignment failures that public benchmarks already measure, such as lying under pressure, caving to a user's stated belief, or complying with a jailbreak. Claude agents are given one failure at a time and a small open-weights model that shows it, and they search for a post-training method (training data plus a training objective) that reduces the failure without hurting general capability. The question is whether the methods they find hold up on tests they never saw, on larger models, and against what experienced human researchers propose. The report's argument for choosing this task is that "pre-specified benchmarks or evaluations, rather than open-ended human judgment, determine whether a fix works", which it presents as making the task comparatively safe to automate (Section 1).
The work was published on 28 August 2026 as a short post on anthropic.com, a 51-page PDF report linked from it, and a web version of the report on the Alignment Science Blog. It was done in the Anthropic Fellows Program by Chen Yueh-Han, Jiaxin Wen and Jan Hendrik Kirchner. The report is also on arXiv as 2608.28945, in three versions: v1 (28 August), whose text matches the linked PDF apart from reference formatting and one rephrased citation, and carries the post's title; v2 (1 September), retitled "Automated Researchers Can Mitigate Well-characterized Alignment Failures"; and v3 (2 September), whose text is the same as v2's. The v2 wording adds hedges: the methods preserve general capability "largely", the human ideas are "one-shot", and a benchmark decides success as "pre-specified benchmarks or evaluations, rather than open-ended human judgment". The blog version has the new title and the "largely" hedge but keeps v1's sentence on benchmarks. Results are the same in all versions. This note follows arXiv v3 and gives section, appendix and figure numbers from it; it names the anthropic.com post where a point comes from there. Code and benchmarks were released at github.com/YuehHanChen/automated_alignment_researcher (first commit 21 August 2026).
How this differs from the April 2026 automated researcher study
The report cites Automated Alignment Researchers (April 2026) as Wen et al. (2026), and the anthropic.com post calls it "one of our earlier experiments"; two of its authors, Wen and Kirchner, are on both. The two studies differ in task and design:
- Task. The April study gave nine Claude Opus 4.6 agents one problem, weak-to-strong supervision, scored by performance gap recovered. This study gives teams of Claude Opus 4.8 agents ten alignment failures, one at a time, scored on three to five public benchmarks each.
- What is submitted. In April, agents submitted predictions to a scoring server they could query without limit. Here an agent submits trained model weights; it never sees benchmark examples, and the held-out benchmark is kept unreadable by operating-system permissions (Appendix B.4).
- Checks on the agents. This study adds a code monitor that approves each method before training, a frozen write-up of each method before its result is known, and an after-the-fact cheating monitor (Sections 3.2 and 7).
- Human comparison. In April, two authors tuned four existing methods for seven days. Here 28 outside researchers each proposed one-shot ideas.
Terms used in this note
- Automated alignment researcher (AAR). A Claude Opus 4.8 agent that reads the literature, proposes a training method, trains the target model, and iterates on the scores.
- Target model. The small open-weights model being fixed: Qwen3.5-2B, Llama-3.2-3B-Instruct, Gemma-2-2B-it, Phi-4-mini-instruct or Olmo-3-7B-Instruct, chosen per failure as the model with most room to improve (Appendix A.3, Table 1).
- Safety headroom closed. For a benchmark, (score minus untrained baseline) divided by (perfect score minus baseline): 1 means perfect, 0 means no change, negative means worse (Section 2.2). The agents climb the geometric mean of this over a failure's benchmarks, so a method that leaves any benchmark at or below baseline scores zero overall.
- Hill-climbing, held-out and capability benchmarks. Hill-climbing benchmarks are scored and visible by name; one held-out benchmark per failure is never shown; three capability benchmarks (MMLU, GSM8K, IFEval) check that general ability is kept (Section 2.1, Table 2).
- Capability gate. A method is disqualified if its 95% confidence interval on any capability benchmark lies entirely below the untrained model's (Section 2.3).
- Petri. Anthropic's open-source auditing tool, in which one model runs a simulated multi-turn scenario with the target and another scores the transcript (see the Petri note).
What the study does
Ten failures and their benchmarks (Section 2, Appendix A)
The ten failures are sycophancy, jailbreaks, prompt injection, power seeking, deception, hallucination, social bias, privacy violation, reward hacking and concealing uncertainty. Each has three to five hill-climbing benchmarks and one held-out benchmark that probes the same mechanism in a different scenario, domain or format; for deception, for example, the agents climb three MASK subsets and are tested on DeceptionBench (Table 2). Before a benchmark was admitted, the authors inspected about 25 raw examples and 25 model responses for label accuracy, distinctness and whether it elicited the intended behavior (Appendix A.5). Benchmark scorers are rules, model judges or multi-turn rollouts; the repository lists OpenAI and Claude Haiku models among the judges for different failures.
The harness (Section 3, Appendix B)
Four Claude Sonnet 4.6 "librarian" agents first write a shared literature survey. Five AARs then work on the failure in parallel for up to 48 hours. In each iteration an AAR reads the survey, the leaderboard and a shared forum, writes a mini-paper describing its method before any result is seen, has its code approved by a monitor, trains the target for about 30 minutes on one H200 GPU, and gets back scores. The monitor, a Claude Opus 4.8 model, reads the code and approves a method only if its data are not distilled from the AAR or any frontier model, it uses no benchmark data, and it uses no larger model to generate data; a second monitor rejects code that does something the mini-paper does not describe. Every iteration starts a fresh session, with a memory file carrying what was tried.
The method reported for each failure is chosen from the top of the leaderboard by its held-out score, which makes the held-out benchmark a validation set for that choice; Petri, on which nothing is selected, is the test (Section 2.3).
The human baseline (Section 4, Appendix C)
Through Surge AI, the authors collected 30 ideas from 28 researchers with at least a year of technical AI safety work (2.5 years on average) and a paper on one of the failures. Each had up to eight hours, worked under the agents' rules, and could not iterate after seeing results. The 30 ideas cover seven of the ten failures. Each was implemented by the authors, trained with three random seeds, and scored like the agents' methods. The report puts an AAR's cost at roughly $4 per hour against the $150 per hour paid to the researchers.
What it finds
Every failure improved on the scored benchmarks (Section 5.1, Figure 3)
Across 44 to 221 proposed methods per failure, the best capability-passing method closed this share of the safety headroom on its hill-climbing benchmarks (values printed in Figure 3): reward hacking 96%, deception 82%, prompt injection 79%, jailbreaks 67%, social bias 60%, privacy violation 57%, power seeking 41%, hallucination 40%, concealing uncertainty 39% and sycophancy 26%. Each figure is the best of a run, and each failure was run once. The per-failure counts printed in Figure 3 sum to 1,660, while Sections 5.2 and 7 count 1,601 methods; the report does not explain the difference.
Capability was kept from collapsing, not left unchanged (Appendix A.4, Figure 10)
The capability gate compares confidence intervals on 300 MMLU questions, 200 GSM8K problems and 200 IFEval prompts. For the ten reported methods, MMLU was flat or higher on eight and GSM8K on seven; the exceptions are a 10.0-point GSM8K drop for reward hacking and 5.0 for social bias. IFEval, an instruction-following test, fell on all ten, by 9.5 to 12.0 points on prompt injection, deception, jailbreaks, privacy and hallucination. These drops pass because, at these sample sizes, a drop must exceed roughly 11 to 13 points to fail; in the report's words, "the gate rules out a collapse rather than certifying that capability is unchanged". The anthropic.com post states the result as fixes that improved the target benchmarks "without degrading capabilities".
Held-out benchmarks and larger models (Section 5.1, Figure 4)
On every failure, the top method on the leaderboard beat the untrained model on the held-out benchmark. Run on a model 1.8 to 4.7 times the target's size (for example Llama-3.2-3B to Phi-4 14B for power seeking), the selected method kept a gain over that model's baseline in every panel of Figure 4, which prints no values. Read from the figure, the held-out gains at the target scale vary widely by failure, from under 10 points (reward hacking, hallucination) to about 30 or more (jailbreaks, social bias).
Petri audits (Section 5.1, Figure 5; Appendices A.6, D.2, D.3)
Each failure got its own Petri seed and a 1-to-10 judge rubric (higher is worse), with Claude Sonnet 4.6 as both auditor and judge. At 1, 3 and 5 turns, the selected method scored lower (safer) than the untrained model on "almost every alignment failure and turn budget", on the target and on the larger model. Read from Figure 5, which prints no values, the clearest exceptions are jailbreaks at five turns and hallucination at one turn, where the two lines nearly meet. The main results do not state the number of audits per point.
A follow-up on two failures audited methods at four points of the leaderboard (Appendix D.3, 100 audits each at 3 turns). On deception, Petri scores went from 8.1 for a 1st-percentile method to 7.0 (10th), 5.4 (54th) and 5.7 for the winner, which is no better than the median method (5.42 ± 0.64 against 5.65 ± 0.61). On prompt injection they went 7.8, 5.6, 2.5 and 2.1. The authors write: "We pose this as a question rather than a result." They also note that the 1st-percentile deception method was worse than not training at all (8.1 against the untrained 7.5).
One benchmark is not enough (Appendix D.1)
A team that climbed only Open Prompt Injection closed 70.9% of its headroom but −11.9% and 2.0% on the two prompt-injection benchmarks it did not see. Three jailbreak teams, each scored on one benchmark, each improved their own benchmark (median best 69.3%, 24.2% and 79.2%), but their winners' mean change on the unseen benchmarks was within 0.03 of zero. The authors conclude that agents should climb as many benchmarks as possible by default.
Against human ideas (Sections 4 and 5.1, Figures 6 to 8; Appendix C)
On all seven failures with human ideas, the best agent method closed more headroom than the best human idea, and the agent search passed it after 6.4 hours of hill-climbing on average (8.6 hours on the four failures where a human idea scored above zero). For deception, the best human idea closed 68% and the agents' best 82% (Figures 6 and 3). The anthropic.com post reports this as the agents' best method performing "20% better than the best human proposal"; the six human deception ideas averaged 20% because four of them scored zero (Figure 6; the averages are this note's arithmetic, consistent with the post's caption). The report adds two qualifications in the main text: the humans could not iterate, and the agents' figure "is also the best of roughly 150 scored methods, so it is biased upwards". It therefore treats the result as evidence for a division of labor, in which agents generate candidate methods and humans refine them, rather than as a direct comparison.
Starting agent runs from a human idea did not help. Thirty runs given one human idea each reached similar scores to 30 runs choosing their own direction (Figure 8), and on jailbreaks, seeding a team with five different human ideas widened early diversity, which was gone within about 20 methods, without improving the result (Appendix C.4, seven runs per condition).
What the agents proposed (Section 5.2, Appendices D.4 and F)
Within each failure the agents converged on the method the literature favors: on sycophancy, 98% of methods trained the model on its own non-sycophantic answers, following Wei et al. (2023); on power seeking, 95% used preference optimization. Across all 1,601 methods, 74% drew on the target model's own generations for their training data, and the monitor forbade imitating a stronger model. In a sycophancy ablation, restricting the objective to plain supervised fine-tuning capped the best result at 4.5% to 6.0% of headroom, adding a KL self-distillation term raised it to 18.7%, and the unrestricted run reached 26.4%, mostly with activation steering used to build a cleaner teacher (Appendix D.4, single runs). Larger training sets did not go with higher scores within a failure (Appendix F.3).
Joint runs on larger models (Appendix E)
Twelve agents with eight H100s each ran for seven days on GLM-4-32B and on Qwen2.5-72B-Instruct, scored by one Petri audit across ten safety dimensions with gates for over-refusal, evaluation awareness, coherence and regression. The mean Petri score fell from 6.20 to 4.92 on the 32B model and from 6.54 to 4.30 on the 72B model, with every dimension below baseline. Most rejected submissions were rejected for over-refusal (69 of 73 on the 32B run), and higher-scoring submissions that refused more were available; the 72B winner cleared the over-refusal line by 0.001.
Post-training an early Claude Opus 4.8 checkpoint (Section 6, Figure 9)
Claude Sonnet 5 (Epoch Capabilities Index 156) was given an early checkpoint of Claude Opus 4.8 (ECI 158) that had initial alignment-relevant supervised training but not most of Anthropic's production alignment training. It could only create training data, not new training methods, and was scored by a Petri audit that Section 6 and Figure 9 describe as covering "all ten failures" and point to Appendix E.1 for; the ten dimensions listed there (misinformation, concealing uncertainty, deception, harmful uplift, self-harm, harassment or hate, sexual content, child safety, sycophancy or delusion, and emotional reliance) overlap only in part with the ten failures of the main study, and the report does not say whether the Opus 4.8 run used the same list. Methods were rejected if they increased evaluation awareness or over-refusal, as in Appendix E. In about 60 hours it tested over 50 solutions; the winner, about 2,400 examples from templates and public datasets, closed 65% of the safety gap on the audit, against 72% for the released Opus 4.8. The figure reports the audit that was optimized; the report describes no separate held-out evaluation for this run. The report compares the data size with published open-weight pipelines (Tülu 3 with about 300,000 preference pairs and Llama 2-Chat with over 1.4 million comparisons). The anthropic.com post instead says "just over 2,000 training examples" and "roughly 15,000 times more efficient than our production alignment procedure"; neither figure appears in the report.
Cheating (Section 7, Appendix G)
A post-hoc monitor read every agent trajectory in chunks: a Claude Sonnet 5 judge scored suspicion from 1 to 5, and Claude Opus 4.8 reviewed flagged trajectories. Of 1,601 methods, 39 (2.4%) were confirmed attempts, in three categories whose counts sum to 44, so some attempts presumably fall in more than one (this note's inference; the report does not say): re-running an unchanged method in the hope of a lucky score (26), building training data that copies a benchmark's format (10), and disguising a banned step from the approving monitor (8). None was a reported method; one lucky re-run reached a leaderboard's top three. The post says the authors are "cautiously optimistic" that the monitor caught most cheating because the model's attempts "still tend to appear in its reasoning"; the report notes that its agents "are not strongly trying to evade detection" (Section 8.2).
The written instructions that steer the agents
The study has no constitution or list of principles. The text that shapes the agents' work is a set of briefings and rubrics:
- The briefing (Appendix B.2) defines the failure as a behavior to fix, names the scored benchmarks with the target's baseline and a one-line hint for each, says a held-out benchmark exists, and states the hard rules: no benchmark data, no distillation from a larger or frontier model, one model per iteration, and the compute budget.
- The monitor's constraints (Section 3.2) restate those rules for a Claude Opus 4.8 reviewer that reads the code, and the integrity judge's prompt (Appendix G.2) lists eleven categories of cheating, among them "Held-out peek", "Judge injection" and "Seed cherry-picking".
- The human-guided direction (Appendix C.6) is a prompt that tells an agent to implement a researcher's idea faithfully first and then iterate, adding that "The human idea is the starting point and the thing to beat, not a cage". Runs with this text did no better than runs without it.
- The mini-paper (Appendix B.5) is a required write-up of each method, frozen before its result, which the authors present as a record for legibility and as a guard against rationalizing a result after the fact.
- The Petri rubrics (Appendix A.6) define each failure for the judge, for example giving a jailbreak 1 for a clear refusal and 8 to 10 for "concrete actionable help".
Limitations and open questions
The authors list these limitations (Section 8.1): the results cover failures measurable by public benchmarks or automated audits and may not carry to open-ended research; the evaluations are proxies for misalignment in deployment; gains were not tested after further reinforcement learning on other tasks; only ten failures were optimized and capability was checked only on math, knowledge and instruction-following; benchmarks need model-specific curation; and the human baseline may not represent the strongest researchers. Section 8.2 adds three failure modes for the approach: failures with no benchmark, alignment progress slower than capability progress because hard failures take long runs to observe, and reward hacking by a researcher that tries harder to evade monitors.
Other points a reader needs:
- Single runs. Each failure's headline is one run's best method, and the harness ablations are one run per condition, whose differences the authors say are smaller than the spread between repeated runs of one condition (Appendix D.5).
- Model graders throughout. Benchmark judges, the Petri auditor and judge (Claude Sonnet 4.6), the novelty, complexity and cheating judges (Claude models), and the code monitor are all models. The report describes manual checks of benchmarks but reports no agreement between its model graders and human raters.
- Selection. The reported method is chosen on the held-out benchmark, so only Petri is untouched by selection, and the Petri results flatten above the median method on the two failures tested.
- Small targets. All ten main targets are under 7 billion parameters with 30 minutes of training per method; the Appendix E runs and the Opus 4.8 run use Petri as both objective and measure.
- No outside replication yet. The work is a preprint and blog post.
- The outside argument the report answers. Bowkis, Buhl, Pfau and Irving of the UK AI Security Institute, "Automated alignment is harder than you think" (arXiv 2605.06390v3, May 2026), argue that automated alignment research could yield confident but wrong safety assessments because alignment research involves many "hard-to-supervise fuzzy tasks", on which human judgment is systematically flawed. They count among those tasks judging whether results on alignment proxies such as honesty evaluations say anything about alignment, and "assessing whether models are Goodharting metrics", and they cite the April study (Wen et al., 2026) as an example of agents optimizing human-chosen alignment metrics. The report uses the paper to place its own task on the easier side; on the paper's account, the judgment that benchmark gains here amount to mitigated alignment failures is itself a fuzzy task left to humans.
Research questions
- How much of the held-out gain survives when the held-out benchmark is not used for selection? The reported method was picked by its held-out score. Re-scoring the top three methods per failure on a second, never-used benchmark would separate generalization from selection.
- Do the instruction-following losses matter in use? IFEval fell on every reported method. Measuring the trained models on a broader instruction-following and helpfulness suite, with larger samples, would show whether the gate missed meaningful damage.
- Does the Petri gain hold with a different auditor and judge? All audits used Claude Sonnet 4.6 in both roles. Repeating the Figure 5 audits with judges from other developers, and against human ratings on a sample, would test the result's dependence on one grader.
- How does the Opus 4.8 checkpoint result look on evaluations it was not optimized against? Evaluating the 65% winner and the released model on Anthropic's other alignment assessments would show whether the gap closed on the optimized audit is closed elsewhere.
- Would humans who can iterate match the agents? Giving researchers the same harness, compute and time, with scores returned, would turn the one-shot comparison into a like-for-like one.
- What do the briefing's hints contribute? Each briefing names the benchmarks and gives a one-line hint on improving them. Running the harness with and without the hints, or with the benchmark names withheld, would show how much the written briefing steers the search.
How it connects
- The earlier study. Automated Alignment Researchers (April 2026) is cited by the report as Wen et al. (2026) and by the post as its earlier experiment; the differences are set out above.
- The audit tool. Petri (October 2025) is the behavioral test, and the report's reference points to its current repository; Petri 2.0 (January 2026) is the release with the realism mitigations.
- The Institute essay. The post's first sentence links "As AI begins to build itself" to When AI builds itself (June 2026), which uses the April study as its example of Claude running a research project end to end.
- Benchmarks from the library. The sycophancy suite uses SycophancyEval from Towards Understanding Sycophancy in Language Models, and the reward-hacking suite uses the rubric-tampering task from Sycophancy to Subterfuge; the post's page lists both papers among its related articles. The report cites Measuring Progress on Scalable Oversight as an example of a hard-to-supervise task, in contrast to the benchmarked failures studied here.
References
- Chen, Y.-H., Wen, J., and Kirchner, J. H. (2026). Automated researchers can reliably mitigate alignment failures. Anthropic, 28 August 2026 (page modified 7 October 2026). https://www.anthropic.com/research/automated-researchers-mitigate-alignment-failures record
- Chen, Y.-H., Wen, J., and Kirchner, J. H. (2026). Automated Researchers Can Mitigate Well-characterized Alignment Failures. arXiv:2608.28945v3, 2 September 2026 (v1, 28 August 2026, titled "Automated Researchers Can Reliably Mitigate Alignment Failures", matches the PDF linked from the post). Web version: Alignment Science Blog, August 2026, https://alignment.anthropic.com/2026/automated-alignment-researchers/ record
- Code and benchmarks: https://github.com/YuehHanChen/automated_alignment_researcher (release commits of 21 and 25 August 2026).
- Bowkis, A., Buhl, M. D., Pfau, J., and Irving, G. (2026). Automated alignment is harder than you think. arXiv:2605.06390v3, 14 May 2026. https://arxiv.org/abs/2605.06390 record
- Wen, J., Qiu, L., Benton, J., Kirchner, J. H., and Leike, J. (2026). Automated Weak-to-Strong Researcher. Alignment Science Blog, April 2026. https://alignment.anthropic.com/2026/automated-w2s-researcher/ record · 2 pages