https://terms-en.ai-term-hub.com/zh/tags/attention/Recent content in Attention on 中文AI术语词典Hugozh-cnSat, 18 Jul 2026 11:44:45 +0000https://terms-en.ai-term-hub.com/zh/terms/transformer/Sat, 18 Jul 2026 10:55:40 +0000https://terms-en.ai-term-hub.com/zh/terms/transformer/<h2 id="definition">Definition</h2> <p>Transformer架构在《Attention Is All You Need》论文中被提出，彻底革新了自然语言处理及更多领域。它使用多头自注意力机制来权衡输入序列中不同部分的重要性。</p> <h3 id="summary">Summary</h3> <p>一种基于自注意力机制的深度学习架构，能够并行而非顺序地处理序列数据。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>自注意力</li> <li>多头注意力</li> <li>位置编码</li> <li>编码器-解码器结构</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>机器翻译</li> <li>文本生成</li> <li>图像识别（ViT）</li> </ul> <h2 id="code-example">Code Example</h2> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#f92672">import</span> torch.nn <span style="color:#66d9ef">as</span> nn </span></span><span style="display:flex;"><span>attention_layer <span style="color:#f92672">=</span> nn<span style="color:#f92672">.</span>MultiheadAttention(embed_dim<span style="color:#f92672">=</span><span style="color:#ae81ff">512</span>, num_heads<span style="color:#f92672">=</span><span style="color:#ae81ff">8</span>) </span></span></code></pre></div><h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/attention_mechanism-%E6%B3%A8%E6%84%8F%E5%8A%9B%E6%9C%BA%E5%88%B6/">attention_mechanism (注意力机制)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/bert-bert%E6%A8%A1%E5%9E%8B/">bert (BERT模型)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/gpt-gpt%E6%A8%A1%E5%9E%8B/">gpt (GPT模型)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/self_attention-%E8%87%AA%E6%B3%A8%E6%84%8F%E5%8A%9B/">self_attention (自注意力)</a></li> </ul>