Paper - Attention Is All You Need

Created: October 30, 2025 | Updated: October 30, 2025 | About these notes

---

• Full title: Attention Is All You Need
• Author(s): Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin
• Year: 2017
• Link: https://arxiv.org/abs/1706.03762
• Relevant for:
  • [[Course - Theories of Deep Learning MT25]]^U
  • [[Course - Computer Vision MT25]]^U
• Links to:
  • [[Notes - Computer Vision MT25, Attention and transformers]]^?

Summary

• many important insights
• positional encoding is important for parallelization

---

Related posts

• [[Course - Theories of Deep Learning MT25]]^U

  (outgoing)
• [[Course - Computer Vision MT25]]^U

  (outgoing)
• [[Notes - Computer Vision MT25, Convolutional neural networks]]^U

  (sim: 0.638)
• [[Lecture - Theories of Deep Learning MT25, II, Why deep learning]]^U

  (sim: 0.657)
• [[Paper - Gradient-based learning applied to document recognition, LeCun]]^U

  (sim: 0.621)
• [[Lecture - Machine Learning MT23, XVI]]^U

  (sim: 0.65)
• [[Lecture - Machine Learning MT23, XV]]^U

  (sim: 0.625)