Attention is all you need

Record

Authors
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser; Ł; Polosukhin, I.
Year
2017
Title
Attention is all you need
Published in
Advances in Neural Information Processing Systems 30
Type
Conference paper
DOI
10.48550/arxiv.1706.03762
arXiv
arXiv:1706.03762
Our note
none yet
Verification
verified: its identifier resolves to a record with the same title and author (checked against OpenAlex, 2026-11-14).
Identifiers found: OpenAlex W2626778328 · DOI 10.48550/arxiv.1706.03762 · arXiv:1706.03762
From the lookup
doi (the pages' own wording is under "As cited" below)
Id
vaswani-2017-attention-is-all-you-need

Cited on

As cited

The pages cite this work in 5 forms.

  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762.
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. Cited: Section 3.1 (residual connections, post-norm), Section 3.2.2 (multi-head attention), Section 3.3 and Equation 2 (feedforward network), Section 6.2 and Table 3 (number of heads).
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. Cited: Section 3.2.1 and Equation 1 (scaled dot-product attention), footnote 4 (variance of $q \cdot k$), Section 3.2.3 (masking), Section 3.5 and Table 3 row E (positional encodings).
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. Cited: Section 3.2.2 (head size 64), Section 5.4 (residual dropout).
  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. Cited: Table 3 caption (per-wordpiece perplexities).

BibLaTeX

@inproceedings{vaswani-2017-attention-is-all-you-need,
  author = {Vaswani, A. and Shazeer, N. and Parmar, N. and Uszkoreit, J. and Jones, L. and Gomez, A. N. and {Kaiser} and {Ł} and Polosukhin, I.},
  title = {{Attention is all you need}},
  booktitle = {Advances in Neural Information Processing Systems 30},
  date = {2017},
  doi = {10.48550/arxiv.1706.03762},
  eprint = {1706.03762},
  eprinttype = {arxiv}
}

Download · JSON with CSL-JSON · the whole bibliography