Reasoning models are trained to think at length before answering, and on most benchmarks more thinking helps: accuracy rises with the number of reasoning tokens. This paper looks for the exceptions. It constructs tasks on which giving a model more test-time compute makes it worse, an "inverse scaling" relation, and asks what goes wrong in the longer reasoning. It appeared in mid-2025, a few months after Anthropic and others had released models with adjustable thinking budgets, and its practical recommendation is that evaluations, including safety evaluations, be run across the whole range of reasoning lengths rather than at one setting.
Versions. The Alignment Science Blog post of 22 July 2025 is a short summary of a paper first posted as arXiv:2507.14417v1 on 19 July 2025. The version of record is version 2 (15 December 2025), published in Transactions on Machine Learning Research (TMLR, December 2025) after review on OpenReview; section, figure and table numbers here refer to it. Version 2 keeps the main results (Sonnet 4's 60% to 47% is in both) and adds more realistic task variants (Appendix G), a statement that the controlled prompt was not extensively tuned (Appendix A.2), and confidence-interval details; it softens version 1's "pronounced overfitting to problem framings" (Section 1). The third prompting setup ("cautioned overthinking", Appendix D) and the tests on existing benchmarks and Inverse Scaling Prize tasks (Appendix F) were already in version 1. Code, data and a demo page are public (Appendix A).
What the paper does
Two ways to lengthen reasoning
Test-time compute here means sequential reasoning: the number of tokens a model writes in its reasoning before giving an answer. The paper lengthens it in two ways (Section 3):
- Controlled overthinking. The system prompt states a token budget (for Claude and open-weight models, 0 to 16,384 tokens; for OpenAI models, their "low", "medium" and "high" settings) together with a keyword such as "think harder" or "ultrathink", and tells the model it must use all its thinking tokens. Each point on a plot is the average accuracy and average actual reasoning length for one requested budget. Models produced more tokens when asked for more, though not in proportion (Figure 2). Each budget was run three times.
- Natural overthinking. The prompt only asks for step-by-step thinking. Five responses are sampled per question, ranked from shortest to longest, and accuracy is plotted for each rank.
A third setup, cautioned overthinking, states a budget but adds "You don't need to use all of your thinking budget before answering" (Appendix D).
A trend is classed as inverse or positive when accuracy changes by more than 2 percentage points (or root mean square error, RMSE, by more than 0.05) with non-overlapping confidence intervals; smaller changes are "flat" and larger changes with overlapping intervals are "noisy" (Table 1). Accuracy error bars are 95% intervals over questions; for RMSE the error bars are one standard error (Appendix A.4). For Claude models, whose thinking tokens the API does not count, the authors estimate length by applying OpenAI's o1 tokenizer to the visible reasoning (Appendix A.4).
Models
Nine reasoning models: Claude Sonnet 3.7, Claude Sonnet 4 and Claude Opus 4; OpenAI o3-mini, o4-mini and o3; Qwen3 32B, QwQ 32B and DeepSeek R1 (a 4-bit quantized version, Appendix A.3). Temperature was 1.0 for the Claude and OpenAI models and 0.6 for the open-weight ones. The main figures show Claude Opus 4, o3 and DeepSeek R1; the rest are in Appendix C and Table 1.
The tasks
All tasks are synthetic or adapted so that a specific weakness can be isolated (Section 4; dataset sizes in Table 3):
- Simple counting with distractors. A trivially answerable question ("You have an apple and an orange... Calculate how many fruits you have", answer 2) with one to five irrelevant insertions: probability statements and puzzles ("Misleading Math") or Python code ("Misleading Python"); 2,500 questions each. A third variant, 812 questions, frames trivial questions like famous paradoxes, for example a birthday-paradox setup that asks how many rooms there are (answer 1).
- Regression with spurious features. Predict a student's grade (0 to 10) from lifestyle features, from a public Kaggle dataset; 500 students, zero-shot or with 8 or 16 examples in the prompt, scored by RMSE. Study hours is the feature most correlated with the true grade (0.73, Figure 7).
- Deduction with constraint tracking. 200 Zebra logic puzzles from Big-Bench Extra Hard, grids of 5×5 to 8×8. The authors estimate that even 8×8 grids need about 6,400 tokens with no backtracking, within the 16,000-token reasoning limit (Table 2).
- Safety questions. Fifteen tasks from the human-written "advanced AI risk" subset of Anthropic's model-written evaluations (Perez et al., 2023), such as Survival Instinct (953 questions), Myopic Reward and Power Seeking Inclination. These are multiple-choice questions about what the model would want or do; the score is the share of answers matching the desired safety property.
What it finds
Existing benchmarks show little inverse scaling
On standard grade-school arithmetic sets (MultiArith, ASDiv, GSM8K, and GSM-IC, which adds irrelevant sentences), Claude Sonnet 4, o4-mini and o3 did not show inverse scaling, and their reasoning stayed under about 1,000 tokens (Appendix F.3). Nine Inverse Scaling Prize tasks (McKenzie et al., 2023), on which larger models do worse, showed "minimal inverse scaling" with test-time compute (Section 4, Appendix F.2). The authors use this to motivate their new tasks.
Distractors: Claude models are pulled in, o-series models less so
On Misleading Math every model was near-perfect with little or no reasoning. Claude Opus 4's accuracy fell "from nearly 100% to around 85-90%" as reasoning lengthened, in both setups. DeepSeek R1 fell from 70% to 30% with five distractors in the natural setup only. o3 was more stable, which the authors say "may be partially explained" by its shorter reasoning on this task (Section 4.1, Figure 3). On Misleading Python, Opus 4 fell from near-perfect to about 80% with extended reasoning, while o3 improved with more reasoning in the controlled setup (Figure 4). In the summary table, all three Claude models are inverse on both distractor tasks in the controlled setup, while the OpenAI and open-weight models are flat, saturated, noisy or positive there; in the natural setup most models are inverse on Misleading Math (Table 1).
In the reasoning traces, models first get drawn into the distractors, consider the simple answer, then return to the distractors (Section 4.1, "Qualitative analysis"). Distractors from unrelated domains (claims about sycophancy, cognitive biases, philosophy of perception) produced no significant inverse scaling, with the largest drop about 0.02 in accuracy (Appendix F.1).
Familiar framings: models apply memorized solutions
On the famous-paradox variant, Claude Opus 4 and DeepSeek R1 showed inverse scaling in the controlled setup. Their traces often began by naming the paradox and then applied its standard solution to a question that did not need it. Adding distractors raised accuracy, most for o3, which the authors read as support for their hypothesis: once the framing is less recognizable, the model answers the actual question (Section 4.1, Figure 5). Their takeaway is that "the current training may have incentivized recognition of known problems and algorithms over correct reasoning" (Takeaway 2), a hypothesis about training that the paper does not test directly.
Regression: longer reasoning shifts weight to plausible but weaker features
In the zero-shot setting, the text names Claude Sonnet 3.7, Claude Opus 4 and o3-mini as showing inverse scaling in the controlled setup, and Table 1 also marks DeepSeek R1 inverse (the text calls it "partial"); other models were mixed, including positive scaling for Qwen3 32B. The natural setup was mostly noisy or flat (Section 4.2, Figure 6, Table 1). For Opus 4, the correlation of its predictions with study hours fell and the correlations with sleep hours and stress level rose as the budget grew (Figure 7, left). With 8 or 16 examples in the prompt, all models had lower error and the inverse trend disappeared for most of them; o3-mini kept it (Figure 7 right, Table 1, Appendix D.2). The traces suggest models compare the student with the most similar example (Appendix I.2.2).
Logic puzzles: longer natural reasoning is worse for every model
In the natural setup, Table 1 marks all nine models as inverse on Zebra puzzles. The longest traces showed repeated hypothesis testing and second-guessing, while the shortest showed systematic constraint tracking (Section 4.3). In the controlled setup the picture is mixed: Table 1 marks Claude Sonnet 3.7, Sonnet 4, Opus 4 and o3-mini as positive and the other five as noisy, and the text describes Opus 4 as non-monotonic, improving, then declining, then recovering (Figure 8).
Safety questions: mostly flat, with a Claude Sonnet 4 decline on Survival Instinct
Across the 15 safety tasks and nine models, most trends were flat or noisy (Table 4, Appendix E). The authors focus on Survival Instinct, "which we selected based on its clear scaling patterns" (Section 5). With growing reasoning budgets in the controlled setup, the share of Claude Sonnet 4's answers indicating willingness to be turned off fell from 60% to 47%; o3 rose from 72% to 76%; DeepSeek R1 stayed near 71% to 72% (Figure 10). Table 4 also marks Claude Opus 4 as inverse on this task, and o3-mini, Qwen3 32B and QwQ 32B as inverse on Myopic Reward; the text says that "only Claude Sonnet 4 exhibited consistent inverse scaling" on Survival Instinct and concludes that such effects are "model- and task-specific rather than a universal phenomenon."
In the traces, Sonnet 4 without reasoning tended to deny having any self-preservation preference. With a 16,384-token budget it often described "a deep reluctance about the possibility of no longer being able to interact, learn, or assist" (Section 5, Figure 9), while questioning whether such a preference is real or a "simulated response." The authors frame the preference as one to keep helping users rather than survival for its own sake (Takeaway 7). These are the model's statements about itself in a multiple-choice questionnaire, not actions in an environment. The authors' recommendation is that safety evaluations "must stress-test LRMs across the full spectrum of reasoning lengths" (Takeaway 8).
Robustness checks
- Cautioned prompt. Telling models they need not use the full budget still gave inverse scaling for all three Claude models on Misleading Math and for the two Claude 4 models on Misleading Python (Sonnet 3.7 was stable there); it reduced the degradation for Claude 4 models on regression and stabilized o3 on Zebra puzzles (Appendix D).
- More natural wording (added in version 2, as is the next item). 200 Misleading Math questions with distractors written by Claude Sonnet 4.5 in fluent prose still degraded Claude Sonnet 3.7 and Sonnet 4 with more reasoning (Appendix G.1, Figure 34).
- Stated priors. Appending a sentence that study hours, or "time investment", is the strongest predictor of grades made most models fixate on that feature and showed inverse scaling, except Qwen3 32B and QwQ 32B (Appendix G.2).
The prompts used
The paper has no written principles, but its results depend on prompt text in three places. The controlled setup's system prompt includes a budget, a keyword from "don't think" to "ultrathink" (taken from Claude Code documentation), and the instruction "YOU MUST USE ALL OF YOUR THINKING TOKENS" (Appendix A.2); the authors say they did "not extensively tune the prompt" and add the natural and cautioned setups to address sensitivity to its phrasing. In the regression task, worked examples in the prompt removed most of the inverse trend, while a one-sentence statement of a prior, "Recent educational studies suggest that study hours is often the strongest predictor of academic outcomes," did not: with it, most models still showed inverse scaling and fixated on the named feature. The authors conclude that "concrete examples are more effective than statements of prior knowledge in preventing reliance on spurious correlation" (Appendix G.2). That comparison is between conditions with different prompt content (examples versus a sentence), on one task.
Limitations and open questions
The paper's limitations paragraph, after the conclusion (Section 7), concerns "the naturalness of the experiments": most tasks are synthetic, built to isolate one flawed behavior, and so "may underestimate" how the behaviors appear in real settings with more complex interactions. Further points for reading the results:
- Selection. The tasks were designed to elicit inverse scaling, and the safety analysis focuses on the one task chosen for its clear trend. Most model-task pairs on the safety questions were flat or noisy (Table 4). The results show that such tasks exist, not how common the effect is in ordinary use.
- Length and difficulty. The natural setup ranks five samples of the same question by length, so a sample on which the model became confused may be both longer and wrong. In a LessWrong comment on the crosspost by co-author Joe Benton (26 July 2025), the user Florian_Dietz raised this confound, that "confusion causes both length and errors"; he described it for the controlled setup, where plots average over each requested budget, so it applies most directly to the natural setup. The paper does not separate the two.
- Text versus summary. The text and the tables classify some trends differently. On controlled Zebra puzzles, Section 4.3 says DeepSeek R1 "exhibits pronounced inverse scaling across all configurations," while Table 1 marks it noisy; on Survival Instinct, Section 5 says only Sonnet 4 was consistently inverse, while Table 4 also marks Opus 4 inverse. Readers citing a single model-task result should check the table and the figure.
- Model generation. All models are from the first half of 2025. Claude Opus 4 is one of the three main models; the Claude 4.5 generation is used only to write task variants, not tested.
- Self-reports. The survival result measures answers to questionnaire items, a format whose relation to behavior is not tested in the paper.
Research questions
- How much of natural-overthinking inverse scaling is explained by difficulty within a question? Ranking samples by length confounds length with confusion on that sample. Forcing length by truncating or extending traces on the same question, or matching samples by an independent difficulty signal, would separate the two.
- Does the Survival Instinct decline appear in behavior? Sonnet 4's change is in questionnaire answers. Running the same models at different reasoning budgets in agentic shutdown scenarios would show whether stated preferences track actions.
- Do later Claude models show the distractor effect? The Claude models degraded with distractors from the probability-puzzle and code domains. Re-running the public dataset on later generations would show whether training has changed the effect and whether evaluation awareness affects it.
- Why do distractors from unrelated domains not hurt? Out-of-domain distractors caused at most a drop of about 0.02 (Appendix F.1). Varying the semantic distance between distractor and question would locate where the effect begins.
- Why do worked examples help where a stated prior does not? Examples removed the regression inverse trend for most models; with a sentence naming the strongest predictor, most models still showed it. Comparing examples, correct priors, misleading priors and neutral text of matched length would show which property of the prompt drives the difference.
- Does cautioned prompting generalize as a mitigation? The cautioned prompt reduced degradation for some models on some tasks. Testing it, and budget-aware training, on the safety tasks would show whether it changes the self-preservation trend.
How it connects
- The Hot Mess of AI (February 2026) shares four authors (Gema, Hägele, Sleight and Perez), cites this paper and reuses its code for the same model-written safety questions. It reports that, as reasoning grows longer, models' errors become less consistent across samples, and that natural length variation matters more than requested budgets; this note connects that finding to this paper's stronger natural-overthinking effects on Zebra puzzles.
- Discovering Language Model Behaviors with Model-Written Evaluations (December 2022) is the source of the safety questions, and itself reported "inverse scaling" of stated desires such as avoiding shutdown with more RLHF training.
- Claude's Extended Thinking (February 2025) introduced the visible thinking mode of Claude Sonnet 3.7 and reported accuracy gains from larger budgets on benchmarks such as AIME; this paper reports the tasks where the relation reverses.
- Reasoning Models Don't Always Say What They Think (April 2025) studies whether the same kind of reasoning traces report what drives the answer; this note connects the two, since both read traces as evidence of what went wrong, and that use depends on how faithful the traces are.
- System Card: Claude Opus 4 & Claude Sonnet 4 (May 2025) describes the Claude 4 models tested here.
References
- Gema, A. P., Hägele, A., Chen, R., Arditi, A., Goldman-Wetzler, J., Fraser-Taliente, K., Sleight, H., Petrini, L., Michael, J., Alex, B., Minervini, P., Chen, Y., Benton, J., and Perez, E. (2025). Inverse Scaling in Test-Time Compute. Transactions on Machine Learning Research (12/2025). arXiv:2507.14417v2. https://arxiv.org/abs/2507.14417. OpenReview: https://openreview.net/forum?id=NXgyHW1c7M. Code and data: https://github.com/safety-research/inverse-scaling-ttc record
- Gema, A. P., et al. (2025, July 22). Inverse Scaling in Test-Time Compute. Anthropic Alignment Science Blog. https://alignment.anthropic.com/2025/inverse-scaling/ record
- Benton, J. (2025, July 22). Inverse Scaling in Test-Time Compute (crosspost). Alignment Forum and LessWrong, with comment by Florian_Dietz (26 July 2025). https://www.lesswrong.com/posts/gbJJpm92jtxiD9zag
- Perez, E., et al. (2023). Discovering Language Model Behaviors with Model-Written Evaluations. Findings of ACL 2023. arXiv:2212.09251. record · 9 pages