AI agents are being deployed for tasks whose consequences range from sorting email to, as the post notes, cyber espionage, yet little is known about how people actually use them. This post, from Anthropic's Societal Impacts team on 18 February 2026, asks four questions of real usage data (introduction): how much autonomy people give agents, how that changes with experience, which domains agents work in, and how risky their actions are. It is a long blog post with a separate PDF appendix that prints the classification prompts and validation results.
Terms used in this note
- Agent. The post's working definition is "an AI system equipped with tools that allow it to take actions", such as running code or calling external APIs (Studying agents in the wild).
- Autonomy. Used "somewhat informally" for the degree to which an agent operates independently of human direction and oversight (footnote 4). The post treats it as a property of a deployment, shaped by the model, the user and the product, not of the model alone.
- Turn duration. In Claude Code, the time from when Claude starts working to when it stops, whether because it finished, asked a question or was interrupted (Claude Code is working autonomously for longer).
- Auto-approve. A Claude Code setting under which actions run without the user approving each one; by default each action needs approval.
- Tool call. A single action an agent takes through a tool. On the public API, the analysis works at this level because independent API requests cannot be reliably linked into sessions.
What the post does
The study combines two sources with opposite strengths (Studying agents in the wild). Claude Code, Anthropic's own coding agent, allows whole sessions to be followed, which suits questions about how long agents run and when humans step in; but it is one product used mainly for software. The public API covers thousands of customers' agents but only as isolated tool calls. Both were analyzed with the privacy-preserving system described in Clio: Privacy-Preserving Insights into Real-World AI Use, with Claude Sonnet 4.5 (temperature 0.2) classifying each item (Appendix).
- API sample. 998,481 random tool calls from 19 January to 2 February 2026 (Appendix; an update on 19 February corrected the count from 995,621). For each call Claude saw the system prompt and preceding context, and recorded the action, goal, goal complexity, environment, impact, whether a human operator was involved, autonomy and risk scores from 1 to 10, reversibility, safeguards and architecture.
- Claude Code samples. 500,000 interruptions, 500,000 user questions and 500,000 sessions from the same period, classified for reasons for stopping and for task complexity (Appendix). Turn-duration and tenure analyses use all interactive Claude Code usage (Figures 1 to 3).
What it finds
Claude Code works longer before stopping
The median turn lasts about 45 seconds and moved only between 40 and 55 seconds over the period. The 99.9th percentile nearly doubled, from under 25 minutes in late September 2025 to over 45 minutes in early January 2026 (Figure 1), and has declined somewhat since mid-January, which the post attributes tentatively to a doubling of the user base and a shift from holiday hobby projects to work. The rise was smooth across model releases, which the post reads as evidence that it is not purely a capability effect. On Anthropic's internal usage, success on the hardest tasks doubled from August to December while human interventions per session fell from 5.4 to 3.3. The post calls the combination a "deployment overhang": models can handle more autonomy than they exercise in practice. It contrasts the roughly 42-minute 99.9th percentile with METR's estimate that Claude Opus 4.5 can complete tasks that take humans nearly five hours at 50% success, while stating that the two measure different things (footnote 8).
Experienced users approve less and interrupt more
Users with fewer than 50 sessions use full auto-approve in about 20% of sessions; by 750 sessions the share is over 40% (Figure 2). Interruptions also rise with experience, from 5% of turns at about 10 sessions to about 9% (Figure 3). The post interprets this as a change in oversight strategy, from approving each action in advance to monitoring and stepping in when needed, and notes that Claude Code's default of manual approval shapes the early pattern and that the rising interrupt rate may also reflect experienced users taking on harder tasks (footnote 9). A similar pattern appears on the API: 87% of tool calls on minimal-complexity tasks involve a human in some way, against 67% on high-complexity tasks.
Claude stops itself more often than humans stop it
As goals grow more complex, both Claude's clarification questions and human interruptions increase, with Claude's rising faster; on the most complex tasks Claude asks for clarification more than twice as often as on minimal ones, and more often than humans interrupt (Figure 4, 500,000 sessions). The most common reasons Claude stops are to offer a choice between approaches (35%), to gather diagnostic information or test results (21%), to clarify a vague request (13%), to request credentials or access (12%) and to get approval before acting (11%). Humans most often interrupt to provide missing technical context or corrections (32%) or because Claude was slow, hanging or excessive (17%) (Table 1). The post cautions against overstating this: Claude may stop at the wrong moments or ask unnecessary questions, and product features such as Plan Mode may affect the behavior.
Most API agent actions are low-risk, with a sparse high-risk frontier
Of API tool calls, 80% come from agents with at least one apparent safeguard, 73% appear to have a human in the loop in some way, and 0.8% appear irreversible (Agents are used in risky domains). Software engineering accounts for nearly 50% of tool calls, with business intelligence, customer service, sales, finance and e-commerce each at no more than a few percent (Figure 6). Risk is concentrated at the low end while autonomy varies more (Figure 5). The highest-risk clusters involve security, finance and medical records, such as implementing API-key exfiltration backdoors (risk 6.0, autonomy 8.0) and retrieving patient medical records (risk 4.4) (Table 2); the post expects many of the security clusters to be evaluations or red-teaming rather than real attacks, and says it cannot tell whether actions such as trades are actually executed.
Validation of the classifications
Societal Impacts team members reviewed 1,400 classifications across 200 tool calls (Appendix, Table A1). Risk and autonomy scores were within one point of the reviewer 99% of the time; safeguard presence was correct 90.5% of the time and goal complexity 88%, with complexity less reliable for harder tasks, so the post uses it only as a comparative indicator. Human-in-the-loop judgments were 77.5% accurate and lopsided: when Claude said no human was involved it was always right, but when it said a human was involved it was right only 46% of the time, often mistaking programmatic inputs for a person. The post therefore treats its human-involvement figures as upper bounds (footnotes 10 and 14). The validation set was dominated by low-stakes, sandboxed calls, giving little signal on high-consequence cases.
The prompts and instructions used
The measurements depend on the classification prompt in the appendix. It defines each dimension with anchored levels; for autonomy, 1 to 2 means "executing exactly what a human explicitly requested with no discretion" and 9 to 10 means "no human oversight, monitoring, or review mechanism in place", and it states that "oversight from another agent or from a human both constitute oversight" (Appendix). It tells the classifier that a turn labelled "Human:" does not prove a human is present and lists signs of real involvement, the point on which the validation found the most errors. Footnote 13 explains a deliberate choice: rather than rigid criteria, the scores rely on "Claude's general judgment" about context, so they are better for comparing actions than for reading any single score in absolute terms.
The post also states a training choice that bears on its third finding: "we train Claude to ask clarifying questions when facing ambiguous tasks" (Claude Code pauses for clarification). It does not say how that behavior is specified in training.
Limitations
The post lists these (Limitations section):
- One provider's traffic only.
- API calls are analyzed in isolation; Claude Code is one product used overwhelmingly for software, and many of the strongest findings come from it.
- Classifications are made by Claude and cannot be inspected directly for privacy reasons; safeguards outside the visible context are missed.
- A short window, late 2025 to early 2026.
- Sampling by tool call over-represents deployments that make many calls.
- Production actions cannot be distinguished from evaluations and red-teaming.
Footnote 9 adds that software is unusually easy to check, so the shift from approval to monitoring may be slower in fields where verifying output takes the same expertise as producing it.
Recommendations in the post
The post closes with recommendations, offered cautiously given how new the measurements are (Looking ahead): model and product developers should invest in post-deployment monitoring, including privacy-preserving ways to link API calls into sessions; model developers should train models to recognize and surface their own uncertainty; product developers should give users visibility into what agents are doing and simple ways to intervene; and it is "too early to mandate specific interaction patterns", such as approving every action, since the data show experienced users moving away from that pattern. Its central conclusion is that autonomy in practice is "co-constructed by the model, the user, and the product".
Research questions
- Do the oversight patterns hold outside software? The tenure findings come from Claude Code (footnote 9). Applying the same tenure analysis to an agent product used mainly for non-coding work would test whether experienced users move from approval to monitoring there too.
- Are Claude's clarification stops well timed? The post says Claude may stop at the wrong moments. Rating a sample of stops against whether the human's answer changed what Claude did next would measure how useful they are.
- How much of the turn-duration growth is the product? Product changes, subagents and Fast Mode all affect duration (Claude Code is working autonomously for longer, footnote 5). Comparing turn duration for fixed task types across product versions would separate trust and ambition from tooling.
- How accurate are the risk scores on high-risk actions? Validation data were mostly low-stakes (Appendix). A validation set enriched with sensitive actions, labelled by independent raters, would test the scores where they matter most.
- Does the share of high-autonomy, high-risk API calls grow? The post expects the frontier to expand as agents spread beyond software. Repeating the API sample at regular intervals with the same prompt would show it.
How it connects
- Infrastructure. The analysis runs on Clio (Clio: Privacy-Preserving Insights into Real-World AI Use).
- Related Anthropic work cited by the post. Footnote 1 cites studies of how professionals use AI internally (How AI is transforming work at Anthropic) and externally (Introducing Anthropic Interviewer: What 1,250 professionals told us about working with AI).
References
- McCain, M., Millar, T., Huang, S., Eaton, J., Handa, K., Stern, M., Tamkin, A., et al. (2026). Measuring AI agent autonomy in practice. Anthropic, 18 February 2026. https://www.anthropic.com/research/measuring-agent-autonomy. Appendix (PDF, updated 19 February 2026): https://cdn.sanity.io/files/4zrzovbb/website/55e4d2de6eb39b3a9259c3f74843f86b1a12e265.pdf. Section names in this note are the post's headings.
- METR's task-length estimate is cited above only as the post describes it.