Record
- Authors
- He, B.; Noci, L.; Paliotta, D.; Schlag, I.; Hofmann, T.
- Year
- 2024
- Title
- Understanding and Minimising Outlier Features in Transformer Training
- Published in
- Advances in Neural Information Processing Systems 37 (NeurIPS 2024)
- Type
- Conference paper
- arXiv
- arXiv:2405.19279
- Our note
- none yet
- Verification
- verified: checked by hand against the source named below (the open indexes miss or misdate it).
Notes: checked by hand against https://proceedings.neurips.cc/paper_files/paper/2024/hash/986292a930c3692168b177a770025ab3-Abstract-Conference.html (see the correction below); the open indexes gave needs-review (title similarity 0.894). - Id
he-2024-understanding-minimising-outlier-features-transformer-training
Corrections applied to this record: fields (cited as the NeurIPS 2024 paper (the parser read the arXiv id as a preprint); source: https://proceedings.neurips.cc/paper_files/paper/2024/hash/986292a930c3692168b177a770025ab3-Abstract-Conference.html); verified (the NeurIPS 2024 version of record is titled "...in Transformer Training", as cited; both arXiv versions (v1, May 2024; v2, the camera-ready, November 2024) are titled "...in Neural Network Training"; source: https://proceedings.neurips.cc/paper_files/paper/2024/hash/986292a930c3692168b177a770025ab3-Abstract-Conference.html).
Cited on
- Privileged Bases in the Transformer Residual Stream read status: not stated
As cited
The pages cite this work in one form.
-
He, B., Noci, L., Paliotta, D., Schlag, I., and Hofmann, T. (2024). Understanding and Minimising Outlier Features in Transformer Training. Advances in Neural Information Processing Systems 37 (NeurIPS 2024). arXiv:2405.19279. (Later work on normalization layers, optimizers and outlier features; locations refer to the NeurIPS version.)
BibLaTeX
@inproceedings{he-2024-understanding-minimising-outlier-features-transformer-training,
author = {He, B. and Noci, L. and Paliotta, D. and Schlag, I. and Hofmann, T.},
title = {{Understanding and Minimising Outlier Features in Transformer Training}},
booktitle = {Advances in Neural Information Processing Systems 37 (NeurIPS 2024)},
date = {2024},
eprint = {2405.19279},
eprinttype = {arxiv}
}