Estimating AI productivity gains from Claude conversations

productivityeconomicseconomic indextime estimationlabor

Tamkin and McCrory have Claude estimate, for 100,000 Claude.ai conversations, how long each task would take a professional without AI and how long it took with AI; the estimates imply tasks of about 1.4 hours and time savings of about 80%, and, aggregated with Hulten's theorem under universal adoption over ten years, a 1.8% annual increase in US labor productivity, an exercise the authors say is not a forecast.

Alex Tamkin, Peter McCrory (Anthropic) · 2025-11-25 · original

Counting which tasks people bring to an AI system says little about how much the AI matters economically: a five-minute report and a five-day report count the same. Anthropic's Economic Index had measured the breadth of Claude's uses across occupations; this research note, by Alex Tamkin and Peter McCrory, tries to measure their depth. It asks Claude to estimate how long the task in each conversation would take a skilled person without AI and how long it took with AI, then aggregates the savings across the US economy.

The note was published on Anthropic's site on 25 November 2025; its PDF version and suggested citation give 5 November 2025. Page and figure numbers below refer to the PDF.

What the note does

Two estimates per conversation

Using Anthropic's privacy-preserving analysis system, Clio, the authors sampled 100,000 Claude.ai conversations from the Free, Pro and Max tiers (p. 5). For each, Claude produced two numbers:

  • Time without AI: the hours a competent professional would need to do the task without AI assistance.
  • Time with AI: how long the user actually spent completing the task with Claude, estimated from the conversation.

Following the Economic Index method, conversations were mapped to tasks in O*NET, the US Department of Labor's database of occupations and their constituent tasks, and each task's time was taken as the median over its conversations (p. 5). Time savings are defined as one minus the ratio of time with AI to time without AI. Task cost is time without AI multiplied by the occupation's average hourly wage from the May 2024 Occupational Employment and Wage Statistics (OEWS).

Validation

The authors ran two checks on whether the estimates carry information (pp. 6 to 8).

  • Self-consistency. Several prompt variants (for example, asking about an "employee with appropriate skills" versus a "human worker" who is "competent in the relevant field") were run on 1,800 conversations each, from users who consented to share them. Log-scale correlations across variants were r = 0.89 to 0.93 (p. 6; Figure 2).
  • External benchmark. On 1,000 software tasks from a public dataset of JIRA tickets in open-source projects, which records both developers' estimates and tracked completion times, the authors compared estimates with actual times (p. 7). Developers reached a Spearman rank correlation of ρ = 0.50 and a Pearson correlation of log values of 0.67. Claude Sonnet 4.5, given only the ticket's title and description, reached ρ = 0.44 and 0.46; with ten examples with true times in the prompt, ρ fell to 0.39 and the log correlation rose to 0.48 (pp. 7 to 8; Figure 3).

The authors read this as showing that Claude's estimates carry "directional information" only slightly worse than developers' own. They also report that Claude's estimates are "much more compressed" than humans' (too long for short tasks, too short for long ones) and more prone to overestimates overall, so true differences between tasks may be larger, and true task lengths slightly shorter, than reported (p. 8). Claude Sonnet 4.5 correlated better than Claude Sonnet 4.

What it finds

Task length, cost and savings

  • Averages. Claude estimates the tasks would take on average about 90 minutes (stated elsewhere as 1.4 hours) without AI, and that AI reduces completion time by about 80% (pp. 2 to 3). The Overview gives the average task's labor cost as $55 (p. 3); the results section gives a median of $54 per conversation (p. 10).
  • Distribution of savings. The median conversation shows an estimated 84% time saving (p. 12), and the median across O*NET tasks is 81% (Figure 6, p. 13). Savings cluster between 50% and 95%, peaking at 80% to 90%. Examples at the extremes are checking diagnostic images (20%) and compiling information from reports (about 95%) (p. 12). In the Overview, healthcare assistance tasks show 90% savings and hardware issues 56% (p. 3).
  • By occupation. Management tasks average 2.0 hours, legal 1.8, education 1.7 and arts and media 1.6, while food preparation, installation and maintenance, and transportation tasks average 0.3 to 0.5 hours (p. 10). Average task costs range from $133 for management and $119 for legal to $8 for food preparation and serving (p. 10). Across occupation groups, average hourly wage and estimated task length correlate at r = 0.8 (Figure 5, p. 12).
  • Examples. Curriculum development tasks estimated at 4.5 hours were done in 11 minutes, with an implied labor cost of $115; writing invoices, memos and similar documents saved 87%; financial analysis tasks costing $31 in wages saved 80% (p. 10).

The authors set these savings beside earlier randomized controlled trials, which typically found smaller effects: 56%, 40%, 26%, 14% and, in one study of experienced open-source developers, negative time savings (p. 13). They suggest the difference may come from work done outside the chat window, which the method cannot see, or from those studies' use of earlier models.

Economy-wide estimate

To aggregate, the note uses Hulten's theorem, which says that to a first approximation the economy-wide gain in total factor productivity (TFP, output growth not explained by more labor or capital) is a weighted sum of the log changes in task-level productivity, each weighted by its Domar weight, the ratio of its gross output to GDP (footnote 2, p. 21). Following the "baseline" approach of Acemoglu (2024), the authors weight each task's log speed-up by the share of the occupation's time that Claude estimates is spent on it, and by the occupation's share of the US wage bill. Tasks not seen in the sample are assigned no gain (p. 14). As an illustration, Claude estimates that programmers spend 23% of their time writing and maintaining code (p. 14).

Assuming current models reach universal adoption over ten years, the estimates imply a 1.8% annual increase in US labor productivity (p. 15; Figure 7). The note compares this with average growth of 2.1% per year since 1947 and 1.8% since 2019, and describes it as roughly doubling the recent rate. With a labor share of income of 0.6, the implied TFP increase is about 1.1% per year (1.08% in the Figure 7 caption) (p. 15). The estimate lies toward the upper end of the range of recent estimates compiled by Filippucci, Gal and Schief (2024) (p. 16; Figure 10). Software developers account for the largest share of the gain (19%), followed by general and operations managers (about 6%), market research analysts and marketing specialists (5%), customer service representatives (4%) and secondary school teachers (3%) (p. 17). Retail, restaurants, healthcare delivery and construction contribute little, mainly because few of their tasks appear in the data.

The authors stress that this is "an exercise", not a prediction: it holds capabilities fixed at the time of sampling and assumes universal adoption. They describe it as possibly an approximate lower bound as models improve, while noting that uneven adoption could reduce gains in the short term (pp. 16 to 17).

Bottleneck tasks

If AI speeds up some of an occupation's tasks but not others, the slower tasks become a larger share of the work. The note's example is a home inspector, whose report writing might speed up while travel to the property does not (p. 17). For software developers, the data show little AI use for coordinating installations or supervising other engineers; for teachers, little for running clubs or enforcing classroom rules (Figure 9, p. 18). The authors connect this to an observation from Aghion, Jones and Jones that growth "may be constrained not by what we are good at but rather by what is essential and yet hard to improve" (p. 18).

The prompts used

The appendix prints the four prompts (pp. 23 to 25). The main estimate asks Claude to "Estimate how many hours a competent professional would need to complete the tasks done by the Assistant", assuming the professional has the domain knowledge, context and tools, with a short reasoning step before a numeric answer. A second prompt estimates the user's time in minutes, listing factors such as the number and complexity of the user's messages, reading time and time implementing suggestions outside the conversation when directly relevant. A third, used for the JIRA benchmark, asks for a bare number of hours. A fourth asks Claude to estimate hours per week that workers in an occupation spend on each O*NET task, normalized afterwards. The self-consistency check is the note's test of how much prompt wording matters: the variants it reports differ in how the hypothetical worker is described, and agree at r = 0.89 to 0.93.

Limitations and open questions

The authors list six limitations (pp. 19 to 20):

  • No real-world validation. Claude cannot see work done after the conversation, including checking and refining its output, and there is no ground truth for these conversations.
  • Task taxonomy. Real jobs involve tacit knowledge, relationships and links between tasks that O*NET lists omit; an end-to-end randomized trial of software features found no time savings.
  • Structural assumptions. The comparison could understate gains (it ignores the cost of hiring and briefing a worker) or overstate them (if AI's work is of lower quality).
  • Restructuring. The largest historical gains came from reorganizing firms around a technology, which the model cannot predict.
  • Innovation. The model ignores AI's possible effect on science and innovation.
  • Limited data. Claude.ai conversations are not representative of all AI use, and people bring the tasks they expect Claude to handle well.

Footnote 1 adds that Claude sometimes gives outlier estimates, such as programming tasks taking years or worth millions of dollars, which the authors dampen by weighting task medians by conversation counts (p. 21). The web version of the note states the labor share as 0.64 in one sentence; the PDF and the web version's own footnote 4 give 0.6, and this note follows the PDF.

Research questions

  1. How much time do users spend after the conversation? The main limitation is unobserved work outside the chat. Pairing conversation-based estimates with time-tracking data for the same tasks, with user consent, would measure it.
  2. Do the estimates hold outside software? Validation used only software tickets. A benchmark with tracked times in another domain, such as legal drafting or curriculum writing, would show whether the correlation carries over.
  3. How sensitive are the estimates to the description of the worker? The variants tested agreed at r = 0.89 to 0.93. Testing more distant descriptions, such as a novice versus an expert, would show how much the framing in the prompt drives the numbers.
  4. Can the compression of estimates be corrected? Claude overestimates short tasks and underestimates long ones. Fitting a calibration on the benchmark and applying it to the main sample would show how much the occupation-level differences change.
  5. Do the estimates move as predicted over time? The authors plan to track them in the Economic Index. Repeating the analysis on later samples, with later models as estimators, would show whether estimated task lengths and savings rise.
  6. How large is the gap between these estimates and randomized trials? The note reports savings well above those in earlier trials. Running the estimator on transcripts from a trial with measured outcomes would compare the two directly.

How it connects

References

  • Tamkin, A., & McCrory, P. (2025). Estimating AI productivity gains from Claude conversations. Anthropic. https://www.anthropic.com/research/estimating-productivity-gains (PDF dated 5 November 2025.)
  • Acemoglu (2024), Aghion, Jones and Jones, Filippucci, Gal and Schief (2024), and the randomized trials are cited in the note and were not read for this note.