Attention Head Superposition

Record

Authors
Jermyn, A.; Olah, C.; Henighan, T.
Year
2023
Title
Attention Head Superposition
Published in
Circuits Updates, May 2023. Transformer Circuits Thread
Type
Web page
Links
https://transformer-circuits.pub/2023/may-update/index.html#attention-superposition
Our note
Circuits Updates – May 2023
Verification
url-ok: the web address responds (the page itself is not checked).
https://transformer-circuits.pub/2023/may-update/index.html#attention-superposition: HTTP 200, checked 2026-11-14
Id
jermyn-2023-attention-head-superposition

Cited on

As cited

The pages cite this work in one form.

  • Jermyn, A., Olah, C., and Henighan, T. (2023). Attention Head Superposition. In Circuits Updates, May 2023. Transformer Circuits Thread. https://transformer-circuits.pub/2023/may-update/index.html#attention-superposition

BibLaTeX

@online{jermyn-2023-attention-head-superposition,
  author = {Jermyn, A. and Olah, C. and Henighan, T.},
  title = {{Attention Head Superposition}},
  organization = {Circuits Updates, May 2023. Transformer Circuits Thread},
  date = {2023},
  url = {https://transformer-circuits.pub/2023/may-update/index.html\#attention-superposition}
}

Download · JSON with CSL-JSON · the whole bibliography