Garcon

A December 2021 Transformer Circuits article by Nelson Elhage and colleagues describing Garcon, Anthropic's internal tool for running interpretability experiments on models too large for one machine; researchers attach "probes" that read or modify activations at named points during a forward pass on a remote server. The tool was described, not released, and the article reports no measurements.

Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan et al. (Anthropic) · 2021-12-22 · original

Mechanistic interpretability needs access to a model's internals: activations at any layer, and the ability to change or remove a component. The article "Garcon", published on the Transformer Circuits Thread on 22 December 2021, asks how to keep the notebook workflow used on small models when a model is split across many GPUs and machines. Its answer is a tool it calls Garçon (the page title spells it without the cedilla), described as "one of the core pieces of infrastructure we've built to enable interpretability research at Anthropic". The article appeared the same day as A Mathematical Framework for Transformer Circuits, and its closing section says it was released with that paper "to provide context on our infrastructure". It is a design description of about 1,900 words. It releases no code and reports no measurements.

Who wrote it and how it is listed

The byline lists 25 authors from Anthropic, with Nelson Elhage marked as the core contributor. Parts of the text are in the first person singular ("As I'll talk about later"). The Transformer Circuits index lists the article under December 2021 with the tags "note, infrastructure" and the description "A description of our tooling for doing interpretability on large models." The article links to a video "covering similar content", which is the YouTube talk "Infrastructure - Garcon [rough early thoughts]" (uploaded 24 December 2021, about 49 minutes) in the playlist that Transformer Circuits Videos describes. This note did not watch the video.

The problem Garcon addresses

The article sets out the gap in its first section, "The problem". Notebook work on small models relies on PyTorch hooks to reach intermediate activations, but "when the model scales beyond a single node, there's no obvious way to translate that workflow." The authors wanted interpretability methods to apply to the largest models Anthropic trains, and to any architecture, without extra setup for each model.

How Garcon works

The article describes the interface from the user's side (section "The core Garçon interface"):

  • Servers. A researcher launches a server by giving only a name and the model's path in Anthropic's distributed storage. Model configuration and the number of GPUs are detected automatically. Servers can also be started from a notebook; these are named after the model path, so re-running the cell reconnects to a running server, and they shut down after one hour of inactivity.
  • Remote models. A client connects to a server by name and gets a remote model object. By convention its name carries an "r" prefix for "remote". The basic call runs a forward pass and returns the logits for every token.
  • Probe points and probes. The model exposes named probe points, places inside the model where a tensor can be read or changed; "the precise set may vary with model architecture and size". A probe attaches a user-written function to a probe point. The function receives a "save context" for storing data and the tensor at that point. It may return a replacement tensor or None to leave the value unchanged, a convention the article says it borrows from PyTorch hooks. Probes work on forward passes and on backward passes that take gradients with respect to a loss the user supplies.
  • Code sent to the server. The article's examples read out an MLP activation and set a single neuron to zero. Probe functions are written on the fly and shipped to the server with the cloudpickle library. Weights are read through the same naming scheme (section "Parameters").

Saved data is kept per connection, so several users can share one server. When probe points live on different machines, Garcon scatters inputs and gathers results without the user's involvement.

Why the interface is built this way

The article calls the probe interface "flexible but somewhat unwieldy" (section "Motivation for the “probe” interface"). It keeps state on the server and takes several steps to read one activation. The authors give three reasons for the design:

  • Capture only what is requested. Saving every intermediate value of a large model would be an enormous amount of data.
  • Reduce on the server. A probe can compute a summary (one token position, or a statistic such as an L2 norm) where the model runs. This saves bandwidth and uses the GPUs in parallel.
  • Keep state on the server. The article's example is collecting "dataset examples" for MLP neurons: running "potentially millions" of training-corpus examples through the model and keeping the top 10 activations for each neuron in a small structure that each layer updates. Returning full activations for every example would, the article says, be "orders of magnitude slower". This is the authors' estimate, not a reported measurement.

A simpler wrapper, easy_garcon, covers single-context analysis, and the authors say they expect to build further layers on the probe interface, which they call an "assembly language" (section "Garçon wrappers").

Implementation and performance

The implementation is "around 1,000 lines of code", built on Anthropic's existing distributed and machine-learning infrastructure (section "Implementation notes"). The server runs one process per GPU in lockstep: rank 0 receives each request, broadcasts it to every rank, and gathers the results. Ordinary Python data is encoded with pickle, but the article says the server makes "an effort" to send large tensors "more directly and efficiently". The article states that it is "not uncommon" for an experiment to move tens of gigabytes (section "Performance"). It gives no benchmarks.

Adoption, as the authors report it

The section "Adoption" states that the team uses Garcon "for essentially all of our interpretability work" and that other teams have adopted it for ad hoc exploration. Even for models that fit on one GPU, the authors report benefits from a consistent interface and from long-running servers that host common models, which cut the "time to first plot" for a new experiment. These are the authors' reports of internal practice, without numbers.

The closing section says the most valuable contribution is less the interface than the "philosophy" behind it, and names two properties: every model Anthropic trains is immediately available to interpretability researchers, and notebook-style exploration runs at the cluster's scale.

Was it released, and is it still used?

Garcon was not released. The closing section says the implementation is "very tied to the particular details of our model code and our infrastructure" and invites researchers at other labs to get in touch about building something similar.

Two of the collection's papers credit it in their author-contribution statements:

  • A Mathematical Framework for Transformer Circuits (December 2021) calls Garcon the software that made its analysis of models possible, created by Nelson Elhage with help from Tom Brown, Sam McCandlish and Chris Olah. It adds that Elhage and Catherine Olsson built an internal website on top of Garcon for viewing dataset examples and editing text while watching activations and attention patterns (Author Contributions, "Infrastructure for Analyzing Models"). Its "Additional Resources" section points to this article and to PySvelte.
  • In-context Learning and Induction Heads (March 2022) says its data collection "made extensive use of Garcon infrastructure by Nelson Elhage" (Author Contributions, "Research").

This note searched seven later Anthropic interpretability publications for the name: Softmax Linear Units (June 2022), Toy Models of Superposition (September 2022), Privileged Bases in the Transformer Residual Stream (March 2023), Towards Monosemanticity (October 2023), Scaling Monosemanticity (May 2024), the June 2024 post on the engineering challenges of scaling interpretability, and HeadVis (May 2026). None names Garcon. The search was not exhaustive. Whether Anthropic still uses Garcon, or replaced it, is not stated in any source read for this note.

Limitations

  • The article is a description by the tool's builders. It reports no performance figures, no user counts and no comparison with other approaches.
  • Without the code, a reader cannot check the description against the implementation.
  • The design reflects 2021 models and infrastructure; the article itself expects the interface to change.

Research questions

  1. What did later interpretability infrastructure keep from Garcon's design? The 2024 engineering post describes distributed shuffles and visualization pipelines for sparse autoencoders. Comparing that post with this article would show whether server-side reduction and remote probes remained the model for working at scale.
  2. How much does server-side reduction save in practice? The article claims full activations would make neuron dataset-example collection "orders of magnitude slower". Timing both approaches on one model and corpus would test the estimate.
  3. How do open-source tools compare with Garcon's interface? Later public libraries also expose hook points for reading and editing activations. A side-by-side of their interfaces against the probe-point design described here would show which ideas spread outside Anthropic, given that Garcon itself was never released.

How it connects

References