Attention

Sat, 18 Jul 2026 09:39:58 +0000

Definition

Attention mechanisms enable models to focus on relevant information when processing inputs, particularly in sequential data like text. By calculating attention scores, the model determines which elements of the input have the most influence on the current prediction. This approach overcomes the limitations of fixed-size context windows in recurrent networks. Self-attention, a core component of Transformers, allows every token to attend to every other token, capturing long-range dependencies and contextual relationships efficiently, thereby significantly improving performance in NLP and computer vision tasks.

Sequence on English AI Terms Dictionary

Attention

Definition