Who in academic social science has started handing analysis work to a coding agent, and do those researchers produce more? The post "Coding agents in the social sciences", published by Anthropic's Societal Impacts team on 27 May 2026, reports the baseline wave of a survey of 1,260 quantitative social scientists fielded in February and March 2026. It is the baseline of a larger study that includes a randomized experiment giving researchers access to Claude Code, with results to come (first section of the post). A coding agent here means a tool such as Claude Code or Codex that writes and runs analysis code with some autonomy, unlike a turn-taking chatbot. The byline lists Thomas Lyttelton, Maxim Massenkoff and Nathan Wilmers as equal-contribution lead authors; the six-page appendix gives Lyttelton's and Wilmers's affiliation as MIT and none for Massenkoff. The version of record is the post with its appendix PDF, which holds the sampling details, the regression specification and a pilot-sample comparison.
How the survey was run
Recruitment and sample
The appendix ("Data description") gives the fielding period as 20 February to 24 March 2026. The authors directly emailed 44,700 academics found on social science department websites at R1 and major Canadian universities, in OpenAlex publication records and in conference programs. Graduate directors and scholarly associations' lists carried further invitations. The appendix does not say how many people those channels reached, so no overall response rate can be computed, and none is reported. A screener restricted the sample to people working with quantitative empirical data and, among doctoral students, to those past their second year who plan an academic career. Invitations told recipients that the study concerned academic workflows and that participants would be eligible for a randomized experiment providing access to Claude Max accounts; completers were also offered a $10 gift card. After the screener and dropping incomplete responses, the analytic sample is 1,260 (appendix, "Data description"). No weighting is described; the shares below are unweighted shares of this sample.
By Table A1, 31% are PhD students, 7% postdocs, 25% assistant, 19% associate and 19% full professors. Sociology (21%), political science (17%), economics (17%), management sciences (16%) and psychology (14%) make up most of the sample, with public health, education and communication at 3 to 7% each. Respondents named their discipline in free text (318 distinct answers), grouped "using a mix of Claude classification and hand coding" (Table A1 notes).
Derived variables
- Gender is inferred from first names with the
gender_guesserPython library; "male" and "mostly_male" become "typically male name", the female categories likewise, and unknown or androgynous names are dropped (appendix). Figure 3 shows 561 typically male and 374 typically female names, so 325 of the 1,260 respondents are outside the gender comparison (this note's arithmetic). - Top-25 institution follows the Nature Index 2025 Leading Institutions ranking for the US and Canada; 28% of respondents are at one of them (appendix).
- Coding agent use is a yes to the weekly-use question below plus a follow-up confirming one of the named tools or Google Antigravity (Figure 1 caption).
The survey questions
The post prints the wording of its main items:
- AI use: "Have you previously used genAI models to aid your research process?" ("Coding agents haven't reached most social scientists").
- Coding agent use: "Do you regularly (more than once a week) use an AI coding assistant integrated into your command line (such as Codex, Cursor, or Claude Code)?" (same section).
- Expected productivity, on a 10-point slider from "less productive" to "more productive": "In terms of writing publishable papers, do you think AI makes social scientists less or more productive" (Figure 6 caption).
- Expected field impact, on a slider from "Worse" to "better": "Do you think AI will make the social sciences worse or better" (Figure 6 caption).
Footnote 1 explains that the agent question was limited to command-line tools because the survey launched a month after Claude Cowork and before OpenAI's Codex app; researchers who use agents only through desktop apps are therefore not counted as agent users. Output measures are self-reported counts over the six months before the survey ("Coding agent users are posting more working papers and sending out more grant proposals, but not submitting more to journals", hereafter the output section). The appendix describes no coding of free-text answers other than the discipline field.
What it finds
Adoption
Of the 1,260 respondents, 81% said yes to the AI-use question and 20% were classified as coding agent users ("Coding agents haven't reached most social scientists"); Figure 1's reference line for overall agent use is labeled 19%. Among agent users, 86% reported Claude Code and 31% Codex. The authors note that this self-selected, interested sample showed adoption of about one in five even two months after public attention to Claude Code and Opus 4.6 began in late December 2025.
By discipline, Figure 1 prints coding agent use of 38% in economics (the text says 39%), 25% in political science, 18% in management sciences, 12% in psychology, 11% in sociology, 6% in public health and communication, and 4% in education; general AI use runs from 91% (economics) to 63% (public health). The authors write that the agent gradient "roughly tracks" the gradient in overall AI use but is steeper ("Adoption is highly uneven").
Disparities by career stage, gender and institution
Figure 2 prints agent use of 27% for PhD students, 28% for postdocs, 19% for assistant, 12% for associate and 9% for full professors, against general AI use falling from 92% to 68%. Figure 3 prints the group comparisons:
| Group | AI use | Coding agent use |
|---|---|---|
| Pre-tenure (n=782) / tenured (n=478) | 87% / 72% | 24% / 11% |
| Typically male name (n=561) / female name (n=374) | 81% / 78% | 22% / 9% |
| Top-25 (n=354) / outside top-25 (n=906) | 83% / 81% | 24% / 17% |
| Private (n=367) / public (n=846) | 84% / 80% | 23% / 17% |
The text states that the coding-agent differences by name gender, university status and private or public status are significant at p < 0.05, and the Figure 3 caption says so of "All differences in adoption between groups"; the post gives no test details. Read as covering the AI-use column, the caption is hard to square with the printed values: by this note's arithmetic, a simple two-proportion test on the rounded shares and printed n gives z of about 1.1 for names (81% against 78%), 0.8 for top-25 (83% against 81%) and 1.7 for private (84% against 80%), short of p < 0.05. The coding-agent gaps clear it (z of about 2.7 for top-25). The post also states that the gender gap in agent use persists among respondents who have tried AI and when comparing within discipline and career stage ("Adoption is highly uneven"). The post does not print those within-group estimates. The post's key-findings figure of 40% for top universities corresponds to 24% against 17%, a ratio of about 1.4 (this note's arithmetic). The authors read the pattern as suggesting "higher inequality, at least in this early period of coding agent adoption".
What researchers use AI for
Among AI users, generating code is the most common use: 97% of agent users and 77% of other AI users (Figure 4, left panel). The other uses, agent users first, are editing prose (87%, 72%), methods advice (77%, 63%), literature review (76%, 60%), drafting prose (54%, 30%) and generating ideas (47%, 32%). Across all AI users, about a third had used AI to draft prose at all; by discipline, drafting prose is common only in economics (50%) and management science (46%) (Figure 4, right panel). Respondents could choose several uses.
Output: a descriptive comparison
Figure 5 of the post compares agent users with non-users on six self-reported outputs, from OLS regressions with fixed effects for career stage (five categories), discipline (nine) and survey week, robust standard errors, and each coefficient divided by the non-users' mean (appendix). The adjusted differences are:
- empirical projects started: +9% (p < 0.05);
- grant proposals submitted: +33% (p < 0.01);
- working papers posted: +73% (p < 0.01), with the widest interval;
- conference submissions: +10%, not significant;
- journal submissions: +3%, not significant;
- journal resubmissions: -6%, not significant.
In levels, the text describes about a quarter of a project and about half a working paper more over six months. The appendix says a Poisson specification gives similar results. Without adjustment (Figure A1), agent users report fewer journal submissions (-19%) and resubmissions (-26%), both significant; the appendix attributes this to where agent users sit across disciplines and career stages.
The authors are explicit about the design: "This comparison is purely descriptive", the two groups differ in ways the survey cannot adjust for, and "Differences should not be interpreted as causal" (output section). They offer two readings of the missing journal effect: the time it takes a paper to reach submission, or agents being more useful for starting projects than for the "last mile" of a paper.
Expectations
On the productivity slider, 88% of respondents were above 5 and half at 8 or above ("Researchers expect AI tools to raise productivity..."). Optimism rises with the number of AI use cases a respondent has adopted and is higher among agent users (Figure 6, which prints no values). 70% rated paper productivity more optimistically than field impact, and few rated field impact higher. In the discussion the authors suggest, as possibilities, that respondents may fear congestion and competition for attention, or uses of AI that worsen selective reporting and incremental research; the survey does not ask about these reasons directly.
How far selection matters: the pilot comparison
Because the invitation mentioned Claude Max accounts, the authors expect respondents to be heavier and more optimistic AI users than non-responders ("Discussion"). The appendix ("Pilot sample") compares an early-February pilot whose invitation described only a study of research workflows. The pilot covered sociology, political science and management, excluded full professors, and lacked the coding-agent question, so it cannot check the 20% figure. For comparability, Table A2 restricts the main sample to the same disciplines and stages (n = 446 against 102 in the pilot):
- AI use: 84.5% (main) against 87.3% (pilot);
- productivity expectation: 7.56 against 7.10, a difference of 0.46 (marked with one star);
- field-impact expectation: 5.68 against 4.75, a difference of 0.93 (three stars);
- literature review as a use: 66.0% against 40.4% of AI users (three stars);
- drafting prose: 35.8% against 27.0%.
The table's notes do not define its significance stars. The post's sampling section describes the respondents as "fairly similar" to this earlier sample, citing Table A2; the appendix's own reading of the same table is that the main sample is more optimistic and uses AI for more tasks. The appendix concludes that the main sample is "composed of more pro-AI respondents" but that the gradients it can measure (coding and editing as the top uses, drafting prose rare, productivity rated above field impact) are similar in the pilot.
Limitations
The "Discussion" section names three: recruitment explicitly for a study on AI and workflows, which likely drew heavier and more optimistic users; output differences that are descriptive only; and counts of projects and papers with nothing on their quality. The authors plan later comparisons with "a clean comparison group" and of content as well as quantity. From the appendix, further limits are that no response rate can be computed, that respondents are confined to the US and Canada, and that gender is a classification of first names, not a self-report.
Research questions
- What did the randomized Claude Code experiment find? The baseline cannot separate the effect of agents from the traits of early adopters; the experiment's results on the same output measures would.
- Does the gender gap survive self-reported gender? Name classification drops about a quarter of the sample and can misclassify. Self-reported gender and printed within-discipline estimates would show the gap's size and robustness.
- How far does the 20% adoption figure depend on recruitment? The pilot lacked the coding-agent question. A wave with a generic invitation and the same item would answer it.
- Do agent users' extra working papers become publications? Journal differences were null at baseline. Following the same respondents' submissions and acceptances would show whether the early-pipeline gap carries through.
- Does the command-line definition undercount agent use? Asking about desktop-app agents would show how many researchers it missed and whether they resemble command-line users.
- Why is optimism about the field lower than about productivity? The post offers possible reasons but did not ask; items on congestion, reviewing load and selective reporting would test them.
How it connects
- AI for scientific discovery: How researchers use Claude (November 2026) shares three authors (Lyttelton, Wilmers and Massenkoff) and also pairs a baseline survey with planned study of AI's effect on research, there for scientists, mainly in the biological and medical sciences, recruited from Anthropic grant applicants, alongside classified usage data. This note connects the two; neither document cites the other in its text.
- Agentic coding and persistent returns to expertise (June 2026), with Massenkoff as a co-author, studies Claude Code sessions directly and infers users' occupations, including Life, Physical, and Social Science. This note makes the connection.
- Introducing Anthropic Interviewer and How AI is transforming work at Anthropic are other Anthropic studies that ask people directly about AI at work, the second on Anthropic's own staff using Claude Code. The post shows both as related-article cards but does not cite them in its text; the connection is this note's.
References
- Lyttelton, T., Massenkoff, M., & Wilmers, N. (2026, May 27). Coding agents in the social sciences. Anthropic. https://www.anthropic.com/research/coding-agents-social-sciences record
- Lyttelton, T., Massenkoff, M., & Wilmers, N. (2026). Appendix: Coding Agents in the Social Sciences [PDF, 6 pp.]. https://assets.claude.com/403415e54964751190003985896630e56829e797.pdf record