When Anthropic launched computer use in October 2024, letting Claude operate a desktop to carry out tasks, it faced a monitoring problem that per-message classifiers handle poorly. Some harms are visible only in aggregate: a request to click a button could be UI testing or part of a click farm defrauding advertisers. Others are not known in advance. A classifier trained on a list of harms reduces each interaction to a score and cannot flag a pattern across interactions or a harm nobody listed, and having people read unflagged traffic raises privacy and scale problems ("Hierarchical summarization"). "Monitoring computer use via hierarchical summarization" (Sumers et al., 2025) describes the approach Anthropic's Safeguards Research Team used instead: summarize each interaction, then summarize the summaries.
The post appeared on the Alignment Science Blog on 27 February 2025 (the date in its suggested citation; the first Wayback Machine capture is the same day). Theodore Sumers led the research and wrote the post; the other authors are Raj Agarwal, Nathan Bailey, Tim Belonax, Brian Clarke, Jasmine Deng, Kyla Guru, Evan Frondorf, Keegan Hankes, Jacob Klein, Lynx Lean, Kevin Lin, Linda Petrini, Madeleine Tucker, Ethan Perez, Mrinank Sharma and Nikhil Saxena. The post is the only version; it describes a production system and calls itself "early research with room to improve and evolve." It is a description of practice with one quantitative validation, not a controlled study.
What the system does
- Interaction summaries. A summarizer LLM compresses each prompt and completion, up to hundreds of thousands of tokens of text and images, into a structured summary of a few hundred tokens covering the user's intent, real-world outcomes and metadata such as languages used. The exact summarizer prompt is not published, to reduce the risk of attacks on the summarizer ("Prompt summarization").
- Usage summaries. Because interaction summaries are orders of magnitude shorter, hundreds fit in one context window. They are aggregated, most often per user, into a structured report that breaks usage into types, analyzes potential harms and cites representative interactions so that reviewers can check the inferences ("Usage summarization").
- Classification for triage. The summarization prompt is extended with harm tiers and examples of harmful behavior for each, so that each summary carries a discrete harm classification, at the interaction level or the usage level; this decides what goes to human review ("Combining summarization and classification").
- Place in the safeguards. At the computer use launch, system prompts and prompt classifiers handled anticipated harms such as creating social media accounts, and hierarchical summarization surfaced concerning patterns and new harms for human review, with enforcement after confirmed violations (Figure 2; "Protecting Computer Use"). Summaries are held under the same access controls as the original text.
- Relation to Clio. The post contrasts it with Clio: Clio summarizes and clusters across many users to find broad patterns, while hierarchical summarization analyzes individual users' interactions in depth to flag potential harms, and is used only by Anthropic's Safeguards team ("Hierarchical summarization").
What it reports
Interaction-summary quality. Members of the Safeguards team compared Claude 3.5 Sonnet (new) summaries of computer use interactions, from internal red-teaming and real use, with the original prompts and completions, rating accuracy (no hallucinated content) and completeness on a four-point scale: "Yes", "Mostly", "Partially", "No" ("Ensuring Summary Quality"). The labels in Figure 3 give 84% Yes, 12% Mostly, 4% Partially and 0% No for accuracy, and 93%, 5%, 2% and 0% for completeness; counting "Mostly" or better as sufficient for harm determination gives the post's 96% and 98%. The post does not state the number of summaries rated, the number of raters, whether they rated independently, or how the "diversified sample" (Figure 3 caption) was drawn.
Usage-summary quality. The post calls this "harder to measure." Reviewers read a usage summary, and where they agreed it indicated potentially violative usage, read the cited interactions and a random sample of others; the authors found the summaries "highly reliable indicators of usage patterns," measured the usage-level harm classification by precision and recall against the resulting human judgments, and prompt-engineered the summarizer to improve it ("Ensuring Summary Quality"). No precision or recall figures are given. In this note's reading, since humans reviewed the underlying interactions only where they agreed with a summary, the workflow checks flagged reports; the post does not say how recall was estimated or how many harmful users the summaries missed.
Unanticipated harms. The post reports that prompting the summarizer to watch for harms beyond those specified let it flag such content, with one example: attempts to buy explosive precursors and components, which the specification did not explicitly cover ("Combining summarization and classification"). This is an example, not a measured rate.
What monitoring showed about computer use. The authors write that hierarchical summarization "has proven effective at catching both attempted and actual misuse," that existing mitigations appear "largely effective" because Claude regularly refuses Usage Policy violations, and that apart from known red-teamers the most harmful behavior seen in the wild was "limited instances of advertisement spam on online marketplaces" ("Insights on Computer Use"). No counts, time window or detection rates are given.
The written specification and prompts
The system is steered by written text in three places, none of them published in its production form. Figure 1, a conceptual overview, prints example text for the two summarizer prompts: the interaction summarizer asks for the human's "Goals" and the "Outcomes" of the interaction, and the usage summarizer asks for "Uses," with "a rough percentage breakdown," and "Harms," citing "UUIDs from 1–5 representative examples" for each potential harm; the post says it does not provide the exact interaction-summary prompt ("Prompt summarization"). The summarizer is prompted with "a specification of potential harms" and told to flag overtly harmful usage even when the specification does not describe it precisely ("Hierarchical summarization"); Figure 2 labels the summarization stage with the Usage Policy. The classification step adds written harm tiers with example behaviors for each. The usage-summary prompt asks for a breakdown of usage types, an analysis of potential harms and citations. The post credits the instruction to look beyond the specification with surfacing the explosive-precursor case, and says the usage-level summarizer was improved by prompt engineering, but it reports no comparison of prompt wordings.
Limitations and open questions
- Little measurement. Apart from the summary-quality ratings, the post reports no detection rates, false-positive rates, sample sizes or comparison with classifier-only monitoring.
- Self-assessment. Anthropic staff rated summaries written by Anthropic's own model of traffic on Anthropic's product, and usage-level validation was folded into the review workflow.
- Attacks on the summarizer. The authors withhold the prompt because the summarizer could be attacked; the post does not test, for example, whether a user can write prompts that make the summary understate harm.
- Scope. The post concerns misuse by users of the computer use API, not misbehavior by the model itself.
The post says the team's current focus is automated measures of summary quality and that it is extending the method to cyber and CBRN risks under the Responsible Scaling Policy, to help experts review usage and refine threat models ("Future directions"). A later Anthropic report in this collection, Investigating unintended model actions in our evaluations and internal use (October 2026), lists more monitoring through safety classifiers and hierarchical summarization among its remediations for internal agents, as recorded in the note on that report; there the method is applied to a model's own actions.
Research questions
- What is the recall of hierarchical summarization? Reviewers checked the underlying interactions only when they agreed a summary indicated violations. Seeding known harmful usage patterns, including click-farm-like aggregate patterns, into monitored traffic and measuring how many are flagged would estimate what the system misses.
- Does summarization lose the signal for subtle harms? Interaction summaries were rated accurate and complete for harm determination, but on a sample of unstated size and composition. Rating a sample stratified by harm type, with independent raters and agreement statistics, would show where summaries drop the evidence.
- How much does the instruction to look beyond the specification add? The explosive-precursor case is one example. Running the summarizer with and without that instruction on the same traffic and comparing what reviewers confirm would measure its effect on both new findings and false alarms.
- Can users manipulate the summarizer? The authors withhold the prompt for this reason. A red-team exercise against the summarizer, measuring how often adversarial interactions produce benign-looking summaries, would test the risk.
- Does the method transfer to monitoring agents rather than users? Later Anthropic work applies it to internal agents. Comparing its catch rates with transcript monitors on a benchmark of agent sabotage would show whether summarization helps or hides the evidence.
How it connects
- Clio. The post positions the method beside Clio: Privacy-Preserving Insights into Real-World AI Use, which also has Claude summarize conversations but clusters them across users for aggregate insight.
- Classifiers. The post cites Constitutional Classifiers as an example of the interaction-level classifiers that summarization complements.
- Computer use. The post links Anthropic's October 2024 announcement of the upgraded models and computer use. The launch is also described in Developing a computer use model and the model card addendum for the upgraded Claude 3.5 Sonnet; these links are the note's.
- Later use. Investigating unintended model actions reports extending hierarchical summarization to internal agents. These links are the note's: SLEIGHT-Bench tests monitors that read whole agent transcripts, and Cost-Effective Constitutional Classifiers via Representation Re-use, which shares two authors (Linda Petrini and Mrinank Sharma), studies the cost of classifier-based monitoring.
References
- Sumers, T., Agarwal, R., Bailey, N., Belonax, T., Clarke, B., Deng, J., Frondorf, E., Guru, K., Hankes, K., Klein, J., Lean, L., Lin, K., Petrini, L., Tucker, M., Perez, E., Sharma, M., and Saxena, N. (2025, February 27). Monitoring computer use via hierarchical summarization. Anthropic Alignment Science Blog. https://alignment.anthropic.com/2025/summarization-for-monitoring/ record
- Tamkin, A., McCain, M., et al. (2024). Clio: Privacy-Preserving Insights into Real-World AI Use. arXiv:2412.13678. Cited as the post cites it. record · 4 pages