Attention Mechanism #
- Queries, Keys, and Values
- Attention Pooling by Similarity
- Attention Pooling via Nadaraya–Watson Regression
- Attention Scoring Functions
- Dot Product Attention
- Convenience Functions
- Scaled Dot Product Attention
- Additive Attention
- Bahdanau Attention Mechanism
- Multi-Head Attention
- Self-Attention
- Positional Encoding
- Code implementation (webinar)