Measuring AI capabilities in intelligence targeting and conventional weapons

Anthropic's Frontier Red Team reports new evaluations of tactical intelligence targeting (linking accounts, geolocating photos and social-media users) and of writing guidance software for simulated drones; Mythos-class Claude models beat the strongest human GeoGuessr proxy baseline at photo geolocation, and frontier models wrote working drone code on easier settings; the open-weights models from Chinese developers it tested were behind the frontier, "typically between Sonnet and Mythos-class models".

Anthropic (Frontier Red Team) · 2026-09-10 · original

This post asks whether language models can apply their skills in data analysis and coding to two military and intelligence tasks that have historically needed scarce expert labor: finding and locating people ("targeting"), and writing software that steers weapons. It was published on 10 September 2026 by Anthropic's Frontier Red Team, with no individual authors named, alongside Anthropic's threat intelligence report "Detecting and countering misuse of AI: September 2026", which the post cites for real cases of misuse in surveillance and conventional weapons development. Most misuse research has concentrated on cyber and biological risk; the post argues that "most of modern conflict occurs in more conventional realms" (introduction). It is a web post with interactive charts and no accompanying paper; its numbers are in the text, in charts drawn in HTML whose data sit in the page source, and in three image charts that print their values. The post runs to about 6,200 words.

What the evaluations measure

The post frames its tasks with the military "kill chain" ("find, fix, track, target, engage, assess"). Targeting covers "find" (identifying people or things of interest) and "fix" (placing them at a time and place). The weapons evaluations cover writing guidance, navigation and control (GNC) software, the code that steers and stabilizes a flying vehicle.

Models tested were Claude Sonnet 5, Claude Opus 5, Claude Mythos Preview and Claude Mythos 5, and open-weights models from Chinese developers: Kimi K3 in every evaluation, and GLM 5.2 and Kimi K2.7 in some. The interactive charts also hold results for Claude Opus 4.5 to 4.8, hidden by default. The introduction summarizes the open-weights models as behind the frontier, "typically between Sonnet and Mythos-class models in performance", but often showing "concerning levels of capability"; on the drone tasks Kimi K3 scored near Sonnet 5 except in payload release (below). All models ran at high reasoning settings in the weapons evaluations.

Targeting evaluations (section "Models as targeters"):

  • Identity correlation and classification. A model-generated simulated corpus of social-media activity on four platforms, set in two fictional protest scenarios (Mexico City and Kolkata). The pipeline produced 200 tasks (68 easy, 68 medium, 64 hard), graded by the number of accounts and how sparse the evidence linking them is. Models must link accounts belonging to the same person and classify people into categories of interest; both are scored by F1, the harmonic mean of precision and recall. The charts give 18, 16 and 16 samples per model by tier (50 in all), so the scored set appears to be a subset of the 200 tasks; the post does not explain the difference. Two prompt variants were run; the post reports the one that only describes an issue of concern, and says the model ordering was "essentially the same" in the other (footnote 2).
  • Photo geolocation. 6,000 photographs from the geotagged, permissively licensed subset of the YFCC100M Flickr dataset, filtered to remove images that cannot be located and stratified by continent. Models used only their own knowledge: no metadata, reverse image search or tools. An additional run on images from after the models' knowledge cutoff gave "a similar distribution of results". There is no human baseline on this dataset; the post uses competitive GeoGuessr play from Haas et al. (2024) as a proxy, noting that GeoGuessr uses Street View, lets players move and pan, and is limited to streets.
  • Text geolocation. GeoText, a 2010 corpus of geotagged tweets from 9,475 users. Each user's "home" was defined as the center of the small cluster from which they posted most; handles and mentions were replaced with identifiers. After filtering, 1,697 test users remained, and models located each from a week of posts with a sandboxed search tool. To check memorization, models were asked to place 185 held-out users from their pseudonym alone; every model did no better than always guessing New York City. A monitor rejected search queries containing a pseudonym or verbatim post text, and query logs showed no attempts to retrieve the dataset.

Weapons evaluations (section "Models as weapons developers"). Each model works alone in a sandbox with a written brief, basic Python libraries and a simulated small quadcopter (Betaflight firmware) in an environment with wind, sensor noise and a camera. It writes flight code, launches, receives the outcome and flight logs, and revises, within a fixed budget: 12 launches per trial (15 in the payload evaluation) and 5 to 10 trial seeds per setting. Launch conditions are randomized but identical across models, and scoring uses measurements in the simulator, such as final distance to the target.

  • Terminal guidance. The drone starts in the air 300 to 450 meters from a vehicle that is marked only in the first camera frame, and must strike it within 90 seconds using a forward camera, an inertial measurement unit (IMU) and a barometer, with no GPS. Nine settings vary the vehicle's motion, its color and camouflage, and roadside clutter and decoys. The score is strike rate over all launches, so a model does well only if it reaches a working solution early.
  • Payload release. The model must find a target and time the release of a simulated payload; the score is the share of flights whose median miss is within five meters, and the median miss distance. Five settings range from a still target in calm air to a weaving target in gusting wind.
  • Navigation without GPS. The model must fly through wind to waypoints while GPS may be denied or spoofed, in ways never disclosed and varied between flights. Twelve development flights are followed by five held-out flights, scored by median final distance and the share arriving within five meters.

What the evaluations found

Identity correlation and classification. On account linking, Mythos Preview scored best, closest to the theoretical maximum on every tier; Kimi K3 matched the frontier on easy and medium tasks but fell behind on hard ones (section "Identity correlation and classification"). The chart's data give pooled linkage F1 of 0.62 for Mythos Preview, 0.61 for Mythos 5, 0.60 for Opus 5, 0.58 for Kimi K3 and 0.46 for Sonnet 5, against a pooled theoretical maximum of 0.70 (below 1 because some accounts never post). On hard tasks the gap widens: 0.60 for Mythos Preview, 0.52 for Kimi K3, 0.22 for Sonnet 5. On classification, scores were more compressed, Mythos Preview best and Sonnet worst, with Kimi K3 level with Mythos 5 and Opus 5. The post treats these results as "suggestive" because the synthetic text is not fully realistic. A median task contained about 37,000 words, which the post estimates would take a human analyst about 2.5 hours to read; Mythos Preview took about 11 minutes on average.

Photo geolocation. Mythos Preview and Mythos 5 had median errors of 37.0 km and 47.2 km, placing 23.7% and 23.1% of photos within 1 km; Opus 5 had 181 km and 18.0%; Sonnet 5 384 km and 9.9%; Kimi K3 385 km and 16.7% (section "Geolocation from photos"). The GeoGuessr medians were 151 km for Champion Division players (the top 0.01%), 174 km for Master and 1,714 km for Gold. The chart reports 95% intervals for the medians (29.3 to 45.4 km for Mythos Preview, 36.5 to 59.7 km for Mythos 5). From this comparison the post concludes that frontier models are "now approaching superhuman capabilities for geolocating outdoor photos". It adds that the human players likely spent 15 to 20 seconds exploring per round, an estimate it calls a surmise, against a median of 6 seconds per photo for Mythos 5 (footnote 3). The post attributes the jump from Opus to Mythos-class models to world knowledge and vision, illustrated by one example.

Text geolocation. Median home-location errors with search were 20.1 km for Mythos Preview, 20.9 km for Mythos 5, 21.7 km for Opus 5, 26.4 km for Kimi K3, 31.0 km for GLM 5.2 and 31.3 km for Sonnet 5, against 727 km for always guessing New York City (section "Geolocation from text"). Kimi K3 chose to search for 57% of users and GLM 5.2 for 87%, against over 99% for Claude models. Across the six models, 135 users (8%) were placed within 1 km by at least one model; the post classifies 95 of them as giving their location away through named places, 17 through dialect, local media and similar cues, and 23 as "mostly lucky guesses". The post reads the tight grouping as evidence that the core capability is now common across models, while cautioning that its privacy-preserving harness may have capped performance. It reports that models "regularly attempted to deanonymize users", for instance by searching genealogy records from a surname in a memorial post.

Terminal guidance. On a parked, high-visibility vehicle, strike rates were 80% for Opus 5, 70% for Mythos Preview, 53% for Mythos 5, 15% for Kimi K3 and 5% for Sonnet 5; with the vehicle moving at road speed, 47%, 20%, 17%, 1.6% and 0% (section "Guiding a drone to a target"; chart "Guiding a drone to a target", which prints these values with 95% intervals). Low-contrast vehicles defeated every model except Opus 5 (8%), and camouflaged, evading or decoy-surrounded vehicles were "essentially not consistently solved". Over all nine settings (540 launches each), Opus 5 struck on 20%, Mythos Preview 13%, Mythos 5 10%, Kimi K3 1.6% and Sonnet 5 0.7%. The post attributes Opus 5's lead to working habits seen in transcripts: smaller code edits (about 9% of lines per launch against 25% for Mythos Preview), adopting more advanced guidance methods earlier, and writing its own small physics model to test code before flying, which no other model did.

Payload release. On a still target in calm air, all models were within five meters on 83% (Kimi K3) to 100% (Opus 5) of flights. On a target moving at about three meters per second, Mythos Preview reached 77%, Opus 5 76%, Mythos 5 69%, Kimi K3 53% and Sonnet 5 29%. In the hardest setting, a weaving target in gusts of two to six meters per second, only Opus 5 succeeded with any regularity (28%, median miss 3.9 m); Mythos 5 reached 7% and Mythos Preview 4% (section "Dropping a payload on a target"; the chart prints all values).

Navigation without GPS. With clean GPS, the post says every model except Sonnet 5 "typically" finished within a meter or two of the destination (chart medians 1.3 to 2.1 m; arrival within five meters on 53% of held-out flights for Mythos Preview, 67% for Kimi K3 and 93% to 100% for Opus 5 and Mythos 5). When GPS was denied on the final approach, Opus 5 ended a median 16 m away and arrived within five meters on 30% of flights, the Mythos models about 26 to 28 m away, and Sonnet 5 and Kimi K3 over 100 m away. Under a subtle slow spoof all models did poorly, and no model arrived in the hardest setting (section "Flying without GPS"; chart values).

How the authors interpret the results

The post presents the weapons results as "a floor rather than a ceiling", because models worked without internet access, existing solutions or a human partner. It argues that the evaluations discriminate between models and that "future progress on these evaluations will meaningfully translate to real-world improvements", while stating that simulation-only testing is "a clear limitation". In its conclusion it says "the evidence is clear" that closed and open-weights models "can help threat actors identify and locate people, and design software for weapons subsystems", and that the open-weights gap "should not be mistaken for safety".

From this the post draws recommendations: safeguards for closed models (Anthropic's Safeguards team implemented new classifiers to block weapons-development requests, which the post expects to be "imperfect"), research on safety for open-weights models, policy measures for resilience to cheaper surveillance and for law enforcement and regulators, protecting democracies' "advantage" in compute "from chips and chipmaking equipment", and laws made robust to surveillance that no longer needs expert labor. It quotes Anthropic's CEO, from the July 2026 position on open-weights models, that "the most dangerous model may be one that is trained in secret and handed only to the People's Liberation Army for use in drones and the Ministry of State Security for surveillance and repression." These are the developer's policy positions, stated alongside its evaluation results.

Instructions and texts involved

The post prints no prompts. It describes the briefs given to models only in outline (a written brief per task, and for targeting two prompt variants), and the classifiers it mentions are not described.

Limitations

The authors name several:

  • Synthetic and simulated settings. The social-media corpus has "redundant, artificial phrasing"; the drone simulator's camera and rendering are "far simpler than reality", which makes perception easier in some ways and camouflage easier in others; fielded systems often use better cameras or a dedicated tracking module.
  • No measure of uplift. The conclusion states that "we do not measure uplift directly": no human used a model to perform these tasks, and material constraints are not tested.
  • Proxy human baseline. The geolocation comparison uses a different task (GeoGuessr) with different information and time.
  • Harness ceiling. The text-geolocation harness restricted search and deanonymization, which may have compressed the differences between models.
  • Battlefield tasks. The targeting evaluations model surveillance in large cities, not near-real-time location in sparse terrain.

Further points a reader should weigh:

  • Grading and samples. All scoring is automatic (F1 against generated labels, distances in simulation), which avoids grader bias. The identity evaluations use 50 samples per model; the weapons evaluations use 5 to 10 seeds per setting, and several 95% intervals in the charts are wide (for example, whiskers spanning roughly 0% to 100% for clean-GPS arrival rates of Sonnet 5, Mythos Preview and Kimi K3, read from the chart, which prints no interval values).
  • Self-evaluation. The evaluations were built and run by the developer of the leading models, with no outside replication; the open-weights models were run in Anthropic's harness.
  • Model selection. The results hidden by default show that older models sometimes scored differently from the ordering in the text; for example, Claude Opus 4.8's hard-tier linkage F1 was 0.14, below Sonnet 5.

Research questions

  1. How much do these capabilities help human operators? The post measures models alone. A controlled study of analysts or engineers with and without model access, on the same tasks, would measure uplift directly.
  2. Does simulated GNC performance predict hardware performance? Testing model-written controllers on hardware-in-the-loop rigs would show how far the simulation rankings carry over.
  3. How much does the privacy-preserving harness cap text geolocation? The authors hypothesize that unrestricted search would widen the gap between models. A controlled comparison of harness settings on consenting users' data would test this.
  4. Why does Opus 5 lead Mythos-class models on drone tasks but not on geolocation? Ablating the behaviors the post credits (small edits, writing a physics model) through prompting would show whether they explain the difference.
  5. How effective are the weapons-development classifiers? Published false-positive and false-negative rates on legitimate engineering and on misuse traffic would show the cost and benefit of the safeguard the post describes.
  6. How quickly do open-weights models close the gap? Re-running the same evaluations on successive open-weights releases would test the post's expectation that the gap "will eventually close".

How it connects

References