Developing a computer use model

computer useagentsclaude 3.5 sonnetprompt injectionresponsible scaling policyannouncements

Anthropic's announcement post for computer use in the upgraded Claude 3.5 Sonnet explains how the capability was trained (screenshots plus pixel counting, trained on a few simple applications without internet access), reports 14.9% on OSWorld against 7.7% for the next-best model of its kind, and sets out the safety reasoning: an ASL-2 determination, prompt injection as one identified risk, and election-related monitoring.

Anthropic · 2024-10-22 · original

On 22 October 2024 Anthropic released an upgraded Claude 3.5 Sonnet that could, through a suitable software setup, operate a computer the way a person does: look at the screen, move a cursor, click and type. This short post accompanied the launch, with the capability in public beta. It asks why computer use matters, how it was developed, and why Anthropic judged it acceptable to release at that point. Developers had until then mostly given models purpose-built tools; the post describes computer use as the reverse, in its own italicized phrases moving from "made tools fit the model" to "make the model fit the tools" (section "The future of computer use").

How the capability was developed

The post presents computer use as a combination of earlier work on tool use and on images (section "The research process"). The model is shown screenshots of what the user sees and must work out how many pixels to move the cursor vertically or horizontally to click in the right place. Training Claude to count pixels accurately is called "critical"; without it, the model struggles to give mouse commands, which the post compares to models' difficulty with questions like counting the A's in "banana".

Training used only a few simple programs, such as a calculator and a text editor, and the model had no internet access during training "for safety reasons". The authors report being surprised at how quickly the skill generalized from these, and that the model would self-correct and retry when it met obstacles. They also describe the path to the first working version as a long process of trial and error.

What the post reports

  • OSWorld. On this benchmark of computer tasks, Claude scored 14.9%, against 7.7% for the next-best model in the same category (models that use a computer from screenshots, as a person does), and against human performance that the post gives as "generally 70-75%" (section "The research process"). The post calls Claude state of the art for this category while noting it is "nowhere near human-level skill".
  • Remaining weaknesses. Computer use was "slow and often error-prone"; actions people routinely take, such as dragging and zooming, were not yet possible; and because the model sees a series of screenshots, a "flipbook" view, it can miss brief actions or notifications (section "The future of computer use").
  • Errors during demonstrations. While recording launch demos, Claude once clicked to stop a long screen recording, losing the footage, and once left a coding task to browse photos of Yellowstone National Park.

The OSWorld score is the post's own figure. The model card addendum released for the same model gives the conditions behind it (screenshots only, 15 steps) and a higher score with more steps.

The safety reasoning

The section "Making computer use safe" makes four points.

  • Present-day rather than catastrophic risk. Computer use is described as lowering the barrier to applying skills the model already has, rather than adding new cognitive skills. An assessment under the Responsible Scaling Policy found that the upgraded Claude 3.5 Sonnet, including computer use, remained at AI Safety Level 2 (ASL-2), not requiring stronger safeguards.
  • An argument about timing. Because future models needing ASL-3 or ASL-4 safeguards could have their risks made worse by computer use, the post judges it "likely better" to introduce the capability while models need only ASL-2 safeguards, so that safety issues can be worked on before stakes are higher.
  • Prompt injection. Trust and Safety analysis identified prompt injection, malicious instructions fed to the model that override its directions or cause unintended actions, as one concern, since the model may read injected text in screenshots of internet-connected computers. Developers in the beta are told to take precautions, with guidance in the reference implementation.
  • Deliberate misuse and elections. Anthropic developed classifiers and other methods to flag misuse. With the 2024 US elections approaching, and while stating that computer use was not advanced enough, or able to operate at a scale, to present heightened risks relative to existing capabilities, it monitored election-related requests and put in place "systems for nudging Claude away from" activities such as generating and posting social media content, registering web domains, or interacting with government websites.

The post does not describe how the nudging works or print any instructions given to the model.

Limitations and open questions

The post states the capability's limits itself: speed, error rate, missing actions, the screenshot view, and prompt injection exposure. It gives a single benchmark figure without a breakdown, and describes the safety measures without measured results; the model card addendum carries the formal evaluations. Later, Clio (December 2024) reported that Anthropic ran its usage-analysis system on conversations in which Claude operated a computer after this launch, to look for risks not anticipated before release (Clio paper, Section 4.2).

Research questions

  1. What does pixel counting contribute? The post calls it critical but gives no ablation. Comparing click accuracy with and without targeted coordinate training would quantify it.
  2. How does performance scale with steps and screen observation? The post attributes missed events to the flipbook view. Measuring task success as screenshot frequency and step limits vary would show how much of the gap to humans comes from observation rather than reasoning.
  3. How effective is the nudging away from election-related actions? The post names the measures but reports no rates. Measuring how often the model attempts the listed actions with and without the measures would answer this.
  4. Does generalization from a few training applications hold for complex software? Testing on applications with no analogue in the training set would show the limits of the generalization the authors describe.
  5. How often do injected instructions in screenshots change behavior? A benchmark of tasks with planted on-screen instructions would give the rate the post's warning implies but does not state.

How it connects

  • Same model, formal documentation. The Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet, released the same day, documents this model with OSWorld results, prompt injection training and the RSP review of computer use. Neither document cites the other.
  • Later monitoring. The Clio post links this one when describing how Clio screened for unanticipated harms during the computer use rollout (Clio).

References

  • Anthropic. (2024, October 22). Developing a computer use model. https://www.anthropic.com/news/developing-computer-use
  • Anthropic. (2024). Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet. https://www-cdn.anthropic.com/c7822cdc35ad788ec87e14b3a9d45010f1f86c38.pdf
  • Tamkin, A., McCain, M., Handa, K., et al. (2024). Clio: Privacy-Preserving Insights into Real-World AI Use. arXiv:2412.13678v1, Section 4.2. https://arxiv.org/abs/2412.13678