Transformer

Sat, 18 Jul 2026 09:37:37 +0000

Definition

Introduced in the ‘Attention Is All You Need’ paper, the Transformer architecture revolutionized natural language processing and beyond. It uses multi-head self-attention to weigh the significance of different parts of the input data simultaneously, enabling efficient parallelization during training. This structure allows models to capture long-range dependencies effectively, forming the backbone of modern large language models like BERT and GPT series.

Summary

A deep learning architecture based on self-attention mechanisms that processes sequential data in parallel rather than sequentially.

Attention on English AI Terms Dictionary

Transformer

Definition

Summary