"Distributed representation" is an old idea in neuroscience and in connectionist AI: a concept is represented by a pattern of activity across many units rather than by one dedicated unit. After Anthropic published Toy Models of Superposition in 2022, researchers often asked how superposition relates to that older idea. This short essay, published on the Transformer Circuits Thread, is Chris Olah's answer. It argues that the term bundles two different ideas, which it calls composition and superposition, and that they are not points on one scale but separate strategies that trade off against each other. The essay labels itself an informal note: "Mistakes are very possible!" ("About"). It also disclaims novelty, saying the distinction may well have been drawn before in neural coding or connectionism and asking readers for references (footnote 2).
The running example: Thorpe's colored shapes
The essay borrows four example codes from Thorpe (1989), who used them to show the range between a "local" and a "distributed" code in neuroscience. Each code represents colored shapes with binary neurons (each either on or off). The essay's diagrams use five colors and three shapes, fifteen objects in all.
- Local code. One neuron for each (color, shape) pair; the diagram uses 15 neurons. In the essay's terms this code is monosemantic with no superposition ("'Local' Codes"). It looks wasteful but is flexible: it can represent any set of objects at once (a red circle and a blue square together), and a downstream neuron can pick out any combination with a linear function, for example firing for black circles and white triangles but not black triangles or white circles. The essay also uses it to make a point about linear probes (simple classifiers trained to read a property from activations): a probe for "red" would succeed on this code, yet the code has no "red" feature, only red-circle, red-triangle and red-square units.
- Semi-local, or compositional, code. One neuron per color and one per shape, 8 in the diagram, with an object represented by two neurons firing together ("'Semi-Local' / Compositional Codes"). This is what machine-learning researchers usually mean by a distributed representation; the essay's example is Mikolov's 2013 finding that word embeddings encode properties such as gender or plural as directions. Its advantages are fewer neurons (n composable features can describe a number of combinations exponential in n), generalization across combinations (data about green circles can help with green squares), and the ability to represent in-between cases such as a purple shape, as a mix of red and blue. Compositional codes are also the easiest to interpret, because each component can be understood on its own.
- Dense, or highly distributed, code. Only 4 neurons for all the objects, "just as 4 bits can represent 16 numbers" ("'Dense' / Maximal Superposition Codes"). The essay calls this maximal superposition. It can represent far more unrelated objects than it has neurons, but at a cost: most colors or shapes cannot be read out with a linear probe, sets of objects cannot be represented, and neither can in-between objects. This code has also quietly gone back to (color, shape) pairs as its features, because, as the next section argues, maximal superposition is only possible without composition.
- Semi-distributed code. Discussed below as a mixture of the two strategies.
Composition and superposition compete
Arranged by how many neurons they use, the codes seem to form a spectrum from local to highly distributed. The essay argues instead that the compositional code and the dense code take two different strategies ("Composition vs Superposition"). The argument is a counting one. With m binary neurons there are exponentially many (2 to the m) possible activity patterns. Features that can combine freely use up that space quickly: n freely composable features need exponentially many patterns in n, so m bits hold only about m such features. If features never combine, the same m bits can hold exponentially many items in superposition, although a nonlinearity is needed to read them back out. Composition and superposition are two ways of spending the same exponential capacity. The essay suggests Toy Models of Superposition can be read as exploring this tradeoff, expressed there in terms of feature sparsity rather than composition.
The essay draws a practical consequence: descriptions of distributed representations sometimes credit them with both generalization across combinations and storing more features than neurons, but these benefits come from different strategies and trade off against each other.
Mixing the two
The strategies are not entirely exclusive ("Mixing Composition and Superposition"). In practice, features in a compositional representation are usually sparse: only a few are active at once, and many are mutually exclusive. Combinations of many features at once are then rare, leaving low-probability regions of the space that can be used to store other features in superposition. The essay notes that this is when the toy-model experiments found superposition becoming possible. A footnote identifies this as an informal restatement of a result from compressed sensing, the mathematics of recovering sparse vectors from fewer measurements, which suggests that if only a fixed number of features can be active together, the amount of superposition can grow exponentially with the number of neurons (footnote 5).
Thorpe's semi-distributed code is the essay's example of mixing. It keeps separate color and shape features, but stores the colors in superposition with each other and the shapes in superposition with each other, using 5 neurons in the diagram. Its units are polysemantic but respond to only a few features each, and the essay suggests reading its features as combinations of neurons, for example green as the sum of two units' activity.
Conclusion and caveats
The essay concludes that "distributed representations" names two different things, superposition and composition, that can be mixed but are in tension, and that distinguishing them matters because they behave differently ("Conclusion"). Its reason for caring is the interpretability program: a compositional representation can be understood through a number of features linear in the size of the representation, rather than an exponential volume of possible states, which the essay calls critical for understanding neural networks.
The essay sets aside a third, trivial sense of "distributed": a representation that is only a rotation of a local one, which matters when there is no privileged basis but does not change the geometry and cannot be shown with binary neurons (footnote 1). Its examples all use binary neurons, which simplify the space of possibilities; the essay notes that its thinking about continuous activations follows the view that features are directions (footnote 3). Footnote 4 records that the author's view of distributed representations was shaped by conversations with Yoshua Bengio around 2013, which emphasized generalization across combinations.
Limitations
- Informal and unreviewed. The essay asks to be read as ideas a colleague is sharing, not as a paper ("About").
- No experiments. The argument rests on small illustrative codes and a counting argument, not on measurements of trained networks.
- Binary units. The examples use on-off neurons; the essay only sketches how the ideas extend to continuous activations.
- No claim of novelty. The author says the distinction may already exist in the neuroscience or connectionist literature (footnote 2).
Research questions
- Do the features found in trained models form compositional codes, local codes, or a mix? Towards Monosemanticity used this essay's vocabulary to ask why it found local-looking token-in-context features. Measuring, for a set of extracted features, whether pairs combine linearly or appear as separate conjunction features would place real representations on the essay's two axes.
- Can the trade-off be measured? The essay's counting argument predicts that more composition leaves less room for superposition. Training toy models on data with controlled amounts of feature co-occurrence and measuring how many features they store per dimension would test it quantitatively.
- Does the essay's point about linear probes hold in practice? A probe can detect "red" in a local code without a red feature existing. Comparing probe accuracy with the presence of a matching dictionary-learning feature across many concepts would show how often probes find properties the model does not represent as single directions.
- How much of the compressed-sensing bound do real networks reach? Footnote 5 suggests superposition can grow exponentially when co-activation is limited. Estimating how many sparse features a layer of given width actually stores would show how close trained models come.
- Has the composition and superposition distinction been drawn before? The author asks for references. A literature search in neural coding and connectionism (for example, work on sparse and distributed codes) would settle the question of priority the essay leaves open.
How it connects
- Toy Models of Superposition (2022) is the paper whose related-work discussion this essay expands, and which the essay describes as exploring the composition and superposition trade-off in terms of sparsity.
- Towards Monosemanticity (October 2023) refers readers to this essay in its related work and uses its vocabulary of local and compositional codes to ask why many of its features are specific token-in-context features rather than combinations of a token feature and a context feature.
- Mapping the Mind of a Large Language Model (2024) cites the essay for distributed representations and describes superposition as a subtype of distributed representation.
- Circuits Updates (May 2023) lists the essay among the team's recent smaller publications.
References
- Olah, C. (2023). Distributed Representations: Composition & Superposition. Transformer Circuits Thread, 4 May 2023. https://transformer-circuits.pub/2023/superposition-composition/index.html. Section names in this note refer to the web version.
- Anthropic. (2023, May 4). Distributed Representations: Composition & Superposition [research page]. https://www.anthropic.com/research/distributed-representations-composition-superposition
- Thorpe, S. J. (1989). Local vs. Distributed Coding. Intellectica, 8 (1989/2), pp. 3-40. https://doi.org/10.3406/intel.1989.873. (Cited by the essay as the source of the four example codes.)