Adam Optimizer Causes Privileged Basis in Transformer LM Residual Stream

Record

Authors
Caples, D.; Neuhaus, R.
Year
2024
Title
Adam Optimizer Causes Privileged Basis in Transformer LM Residual Stream
Published in
LessWrong
Type
Web page
Links
https://www.lesswrong.com/posts/yrhu6MeFddnGRSLtQ/adam-optimizer-causes-privileged-basis-in-transformer-lm
Our note
none yet
Verification
url-ok: the web address responds (the page itself is not checked).
https://www.lesswrong.com/posts/yrhu6MeFddnGRSLtQ/adam-optimizer-causes-privileged-basis-in-transformer-lm: HTTP 200, checked 2026-11-14
Id
caples-2024-adam-optimizer-causes-privileged-basis-transformer

Cited on

As cited

The pages cite this work in one form.

  • Caples, D., and Neuhaus, R. (2024, September 6). Adam Optimizer Causes Privileged Basis in Transformer LM Residual Stream. LessWrong. https://www.lesswrong.com/posts/yrhu6MeFddnGRSLtQ/adam-optimizer-causes-privileged-basis-in-transformer-lm

BibLaTeX

@online{caples-2024-adam-optimizer-causes-privileged-basis-transformer,
  author = {Caples, D. and Neuhaus, R.},
  title = {{Adam Optimizer Causes Privileged Basis in Transformer LM Residual Stream}},
  organization = {LessWrong},
  date = {2024},
  url = {https://www.lesswrong.com/posts/yrhu6MeFddnGRSLtQ/adam-optimizer-causes-privileged-basis-in-transformer-lm}
}

Download · JSON with CSL-JSON · the whole bibliography