Record
- Authors
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser; Ł; Polosukhin, I.
- Year
- 2017
- Title
- Attention is all you need
- Published in
- Advances in Neural Information Processing Systems 30
- Type
- Conference paper
- DOI
- 10.48550/arxiv.1706.03762
- arXiv
- arXiv:1706.03762
- Our note
- none yet
- Verification
- verified: its identifier resolves to a record with the same title and author (checked against OpenAlex, 2026-11-14).
Identifiers found: OpenAlex W2626778328 · DOI 10.48550/arxiv.1706.03762 · arXiv:1706.03762 - From the lookup
- doi (the pages' own wording is under "As cited" below)
- Id
vaswani-2017-attention-is-all-you-need
Cited on
- Assembling the Transformer Block for Section 3.1; Section 3.2.2; Section 3.3; Section 6.2 read status: not stated
- BPE Tokenization for the Shakespeare GPT read status: not stated
- Building Self-Attention Step by Step for Section 3.2.1; Section 3.2.3; Section 3.5 read status: not stated
- Introduction to Artificial Intelligence read status: not stated
- Language Models from Scratch read status: not stated
- Natural Language Processing read status: not stated
- Scaling the Shakespeare GPT for Section 3.2.2; Section 5.4 read status: not stated
As cited
The pages cite this work in 5 forms.
-
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762.
-
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. Cited: Section 3.1 (residual connections, post-norm), Section 3.2.2 (multi-head attention), Section 3.3 and Equation 2 (feedforward network), Section 6.2 and Table 3 (number of heads).
-
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. Cited: Section 3.2.1 and Equation 1 (scaled dot-product attention), footnote 4 (variance of $q \cdot k$), Section 3.2.3 (masking), Section 3.5 and Table 3 row E (positional encodings).
-
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. Cited: Section 3.2.2 (head size 64), Section 5.4 (residual dropout).
-
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. Cited: Table 3 caption (per-wordpiece perplexities).
BibLaTeX
@inproceedings{vaswani-2017-attention-is-all-you-need,
author = {Vaswani, A. and Shazeer, N. and Parmar, N. and Uszkoreit, J. and Jones, L. and Gomez, A. N. and {Kaiser} and {Ł} and Polosukhin, I.},
title = {{Attention is all you need}},
booktitle = {Advances in Neural Information Processing Systems 30},
date = {2017},
doi = {10.48550/arxiv.1706.03762},
eprint = {1706.03762},
eprinttype = {arxiv}
}