Record
- Authors
- Elhage, N.; et al.
- Year
- 2022
- Title
- Softmax Linear Units
- Published in
- Transformer Circuits Thread
- Type
- Web page
- Links
- https://transformer-circuits.pub/2022/solu/index.html
- Our note
- Softmax Linear Units
- Verification
- url-ok: the web address responds (the page itself is not checked).
https://transformer-circuits.pub/2022/solu/index.html: HTTP 200, checked 2026-11-14 - Id
elhage-2022-softmax-linear-units
Cited on
- Mechanistic Interpretability, Variables, and the Importance of Interpretable Bases read status: not stated
- Softmax Linear Units read status: not stated
- Transformer Circuit Videos for Sections 6.3 read status: not stated
As cited
The pages cite this work in 3 forms.
-
Elhage, N., Hume, T., Olsson, C., Nanda, N., Henighan, T., Johnston, S., El Showk, S., Joseph, N., DasSarma, N., Mann, B., Hernandez, D., Askell, A., Ndousse, K., Jones, A., Drain, D., Chen, A., Bai, Y., Ganguli, D., Lovitt, L., Hatfield-Dodds, Z., Kernion, J., Conerly, T., Kravec, S., Fort, S., Kadavath, S., Jacobson, J., Tran-Johnson, E., Kaplan, J., Clark, J., Brown, T., McCandlish, S., Amodei, D., and Olah, C. (2022). Softmax Linear Units. Transformer Circuits Thread. https://transformer-circuits.pub/2022/solu/index.html. The page gives its publication date as 27 June 2022; Anthropic's research page lists the item under 17 June 2022. Section and figure numbers in this note refer to the web version.
-
Elhage, N., Hume, T., Olsson, C., Nanda, N., et al. (2022). Softmax Linear Units. Transformer Circuits Thread. https://transformer-circuits.pub/2022/solu/index.html. Sections 6.3 ("Qualitative Exploration of SoLU Models") and 7.1 ("Understanding Transformer MLPs").
-
Elhage, N., et al. (2022). Softmax Linear Units. Transformer Circuits Thread. https://transformer-circuits.pub/2022/solu/index.html
BibLaTeX
@online{elhage-2022-softmax-linear-units,
author = {Elhage, N. and others},
title = {{Softmax Linear Units}},
organization = {Transformer Circuits Thread},
date = {2022},
url = {https://transformer-circuits.pub/2022/solu/index.html}
}