Anthropic Education Report: The AI Fluency Index

AI fluencyeducationuser behaviordeployment dataartifactsprivacy-preserving analysis

Anthropic's Education team classifies 9,830 multi-turn Claude.ai conversations from one week in January 2026 for 11 observable behaviors from the 4D AI Fluency Framework, finding that iteration and refinement goes with roughly double the other fluency behaviors, and that conversations producing artifacts show more goal-setting and formatting instructions but less fact-checking, questioning of reasoning and spotting of missing context.

Kristen Swanson, Drew Bent, Zoe Ludwig, Rick Dakan, Joe Feller · 2026-02-23 · original

Most measures of AI adoption count users or tasks. This report asks a different question: as AI becomes part of everyday life, are people developing the skills to use it well? Earlier Anthropic Education Reports described how university students and educators use Claude; this one tries to measure "fluency" itself, by tracking whether a set of behaviors associated with effective and safe collaboration appears in real conversations. It presents the result as a baseline, the AI Fluency Index, for tracking change over time. The post was published on 23 February 2026 and is filed under Announcements; Kristen Swanson designed the research, led the analysis and wrote the report (Acknowledgements). Its anthropic.com address now redirects to a page on Anthropic's learning site.

Terms used in this note

  • 4D AI Fluency Framework. A framework developed by Rick Dakan and Joseph Feller with Anthropic, which the report uses to define 24 behaviors that "exemplify safe and effective human-AI collaboration" (Measuring AI fluency). The post's charts group the behaviors studied here under three competencies, labelled description, delegation and discernment; the discernment behaviors are the ones about evaluating Claude's output (checking facts, questioning reasoning, identifying missing context).
  • Observable behaviors. 11 of the 24 can be seen in a Claude.ai or Claude Code conversation. The other 13, such as being honest about AI's role in one's work or considering the consequences of sharing AI output, happen outside the chat.
  • Iteration and refinement. Building on previous exchanges to improve the work, rather than accepting the first response and moving on.
  • Artifacts. Outputs Claude builds in Claude.ai, such as code, documents and interactive tools.

What the report does

Using the privacy-preserving system described in Clio: Privacy-Preserving Insights into Real-World AI Use, the team sampled 9,830 conversations with several back-and-forth exchanges from a 7-day window in January 2026, and ran 11 binary classifiers, one per behavior, with Claude Sonnet 4 (Claude Haiku 3.5 detected language) (Measuring AI fluency; footnote 1). A screener first removed greetings, single-word exchanges, test messages and chitchat; a manual check of 200 screened-out chats found none that showed any fluency behavior. As a reliability check, behavior rates were compared across days of the week and across six languages; most behaviors varied by 1 to 5 percentage points by day and by 3 points or fewer by language (footnote 2). Footnote 2 dates the sample to 20 to 26 January 2025, while the main text places the window in January 2026.

What it finds

How common each behavior is

The post's chart of the 11 behaviors ranks iterating and refining highest (85.7% of conversations), followed by clarifying the goal before asking for help (51.1%) and providing examples of what good output looks like (41.1%). Specifying format and setting how Claude should interact were each at 30%. The three discernment behaviors ranked seventh, ninth and eleventh of the 11: identifying when the AI might be missing context (20.3%), questioning when its reasoning does not hold up (15.8%) and checking facts and claims that matter (8.7%). Consulting the AI on an approach before execution appeared in 10.1% (chart of behavioral indicator prevalence, Measuring AI fluency).

Iteration goes with the other behaviors

The 8,424 conversations with iteration and refinement showed, on average, 2.67 additional fluency behaviors, about double the 1.33 in the 1,406 conversations without it (Results). The gap was largest for discernment: users in iterative conversations were 5.6 times as likely to question Claude's reasoning and 4 times as likely to identify missing context. The report links this to its earlier Economic Index finding that augmentative use, treating AI as a thought partner, is the most common pattern (Anthropic Economic Index: New building blocks for understanding AI use).

Artifacts: more direction, less evaluation

In the 12.3% of conversations that produced artifacts (1,209 conversations), users were more likely than in the 8,621 others to clarify their goal (+14.7 percentage points), specify a format (+14.5), provide examples (+13.4) and iterate (+9.7). They were less likely to identify missing context (-5.2), check facts (-3.7) and question Claude's reasoning (-3.1) (Results). The report offers three possible explanations without choosing among them: polished outputs may look finished; artifact tasks may depend more on appearance or function than on factual precision; or users may evaluate artifacts outside the conversation, for example by running the code. It connects the pattern to two other Anthropic studies: the Economic Index finding that Claude struggles most on the most complex tasks (Anthropic Economic Index report: Economic primitives) and the coding-skills experiment (How AI Impacts Skill Formation).

Advice to users

A boxed section gives three suggestions drawn from the data: stay in the conversation and treat the first response as a starting point; question polished outputs; and set the terms of the collaboration, since only 30% of conversations include the user telling Claude how to interact. Its examples of such instructions are "Push back if my assumptions are wrong" and "Tell me what you're uncertain about" (Developing your own AI fluency). The report recommends that users write such instructions but does not measure whether they change Claude's responses.

Limitations

The report lists these (Limitations):

  • Sample. Multi-turn Claude.ai users in one week, likely skewed toward early adopters; no seasonal or longitudinal view; no other platforms.
  • Partial framework. Only 11 of 24 behaviors, and none of those about responsible use of outputs.
  • Binary labels. Each behavior is present or absent, missing partial demonstrations.
  • Implicit behaviors. Users may check claims mentally or test artifacts elsewhere, which the conversation does not show.
  • Correlation. The links between behaviors may reflect a common factor such as task complexity or user preference.

The report gives no accuracy figures for its 11 classifiers.

Research questions

  1. Does fluency grow with experience? The report plans cohort analyses comparing new and experienced users (Looking ahead). Comparing behavior rates by account age, controlling for task, would answer it.
  2. Does encouraging iteration cause more critical evaluation? The authors name this causal question. An experiment prompting some users to iterate, and measuring discernment behaviors, would test it.
  3. Do users of artifacts evaluate them outside the chat? The report raises this possibility. Product telemetry on whether generated code is run or edited before reuse, where privacy permits, would show whether lower in-chat discernment means less evaluation overall.
  4. Does setting the interaction mode change Claude's responses? The report recommends instructions such as "Push back if my assumptions are wrong". Comparing Claude's responses in conversations with and without such instructions on similar tasks would show whether they have the intended effect.
  5. How reliable are the 11 classifiers? No validation figures are given. Agreement with human raters on a labeled sample would let readers weigh the small differences, such as the 3.1-point drop in questioning reasoning.

How it connects

References

  • Swanson, K., Bent, D., Ludwig, Z., Dakan, R., and Feller, J. (2026). Anthropic Education Report: The AI Fluency Index. Anthropic, 23 February 2026. https://www.anthropic.com/research/AI-fluency-index (as of November 2026 this address redirects to https://academy.claude.com/tutorials/the-ai-fluency-index; the post's own citation block gives the date as 16 February 2026). Section names in this note are the post's headings; behavior percentages not given in the text are read from the post's charts.
  • Dakan, R., and Feller, J. The 4D AI Fluency Framework, as described in the report and Anthropic's course at https://anthropic.skilljar.com/ai-fluency-framework-foundations (not read for this note).