Transformers on 中文AI术语词典https://terms-en.ai-term-hub.com/zh/tags/transformers/Recent content in Transformers on 中文AI术语词典Hugozh-cnSat, 18 Jul 2026 11:44:45 +0000XLM-RoBERTahttps://terms-en.ai-term-hub.com/zh/terms/xlm_roberta/Sat, 18 Jul 2026 11:38:27 +0000https://terms-en.ai-term-hub.com/zh/terms/xlm_roberta/<h2 id="definition">Definition</h2> <p>XLM-RoBERTa（跨语言语言模型 RoBERTa）是由 Meta AI 开发的大规模多语言模型。它通过在一个涵盖 100 多种语言的多样化数据集上进行预训练，扩展了 RoBERTa 的架构，从而实现了强大的跨语言理解和生成能力，</p> <h3 id="summary">Summary</h3> <p>一种基于 RoBERTa 的多语言 Transformer 模型，在超过 100 种语言的庞大文本数据集上进行预训练。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>多语言自然语言处理</li> <li>Transformer 架构</li> <li>跨语言迁移</li> <li>预训练</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>多语言情感分析</li> <li>跨语言问答</li> <li>机器翻译</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/bert-%E5%8F%8C%E5%90%91%E7%BC%96%E7%A0%81%E5%99%A8%E8%A1%A8%E7%A4%BAtransformer/">BERT (双向编码器表示Transformer)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/roberta-%E4%BC%98%E5%8C%96%E7%89%88%E7%9A%84bert/">RoBERTa (优化版的BERT)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/m-bert-%E5%A4%9A%E8%AF%AD%E8%A8%80bert/">M-BERT (多语言BERT)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/sentence-transformers-%E5%8F%A5%E5%AD%90transformer/">Sentence Transformers (句子Transformer)</a></li> </ul>前缀微调https://terms-en.ai-term-hub.com/zh/terms/prefix_tuning/Sat, 18 Jul 2026 11:30:18 +0000https://terms-en.ai-term-hub.com/zh/terms/prefix_tuning/<h2 id="definition">Definition</h2> <p>前缀微调是一种用于预训练变换器的参数高效适应技术。与更新所有模型权重不同，它在输入序列前prepend（前置）一系列可训练的连续向量（即前缀），从而冻结主干网络并仅优化少量参数以适应新任务。</p> <h3 id="summary">Summary</h3> <p>一种参数高效的微调方法，通过在变换器层的输入端添加可训练的连续向量来实现适配。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>参数高效微调</li> <li>软提示</li> <li>变换器层</li> <li>冻结主干</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>少样本学习适配</li> <li>资源受限下的多任务学习</li> <li>为利基领域定制大型语言模型</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/prompt_tuning-%E6%8F%90%E7%A4%BA%E5%BE%AE%E8%B0%83/">prompt_tuning (提示微调)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/p_tuning-p-tuning/">p_tuning (P-Tuning)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/adapter_modules-%E9%80%82%E9%85%8D%E5%99%A8%E6%A8%A1%E5%9D%97/">adapter_modules (适配器模块)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/peft-%E5%8F%82%E6%95%B0%E9%AB%98%E6%95%88%E5%BE%AE%E8%B0%83%E6%8A%80%E6%9C%AF/">peft (参数高效微调技术)</a></li> </ul>长上下文https://terms-en.ai-term-hub.com/zh/terms/long_context/Sat, 18 Jul 2026 11:24:46 +0000https://terms-en.ai-term-hub.com/zh/terms/long_context/<h2 id="definition">Definition</h2> <p>长上下文指的是基于 Transformer 的模型处理极长输入长度的能力，通常超过标准的 2k 或 4k 标记限制。这种能力使模型能够分析完整的文档、代码库或长文本，保持全局一致性和细节记忆。</p> <h3 id="summary">Summary</h3> <p>语言模型处理并保留包含数千或数百万个标记（token）的输入序列信息的能力。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>上下文窗口</li> <li>标记限制</li> <li>注意力机制</li> <li>位置编码</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>总结完整的法律合同</li> <li>分析完整的源代码仓库</li> <li>处理长篇音频转录文本</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/context-window-%E4%B8%8A%E4%B8%8B%E6%96%87%E7%AA%97%E5%8F%A3/">Context Window (上下文窗口)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/transformer-architecture-transformer-%E6%9E%B6%E6%9E%84/">Transformer Architecture (Transformer 架构)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/rope-rotary-positional-embeddings-%E6%97%8B%E8%BD%AC%E4%BD%8D%E7%BD%AE%E7%BC%96%E7%A0%81/">RoPE (Rotary Positional Embeddings, 旋转位置编码)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/kv-cache-%E9%94%AE%E5%80%BC%E7%BC%93%E5%AD%98/">KV Cache (键值缓存)</a></li> </ul>掩码填充https://terms-en.ai-term-hub.com/zh/terms/fill_mask/Sat, 18 Jul 2026 11:17:25 +0000https://terms-en.ai-term-hub.com/zh/terms/fill_mask/<h2 id="definition">Definition</h2> <p>掩码填充是 BERT 等基于 Transformer 模型中使用的一种基本预训练目标。该过程涉及掩盖文本序列中的随机标记，并训练模型预测被掩盖的原始标记，从而学习语言的深层语义表示。</p> <h3 id="summary">Summary</h3> <p>一种自然语言处理任务，模型根据上下文预测句子中缺失的标记。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>掩码语言建模</li> <li>上下文理解</li> <li>自监督学习</li> <li>标记预测</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>文本补全</li> <li>语义角色标注</li> <li>预训练基础</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/bert-%E5%8F%8C%E5%90%91%E7%BC%96%E7%A0%81%E5%99%A8%E8%A1%A8%E7%A4%BA/">BERT (双向编码器表示)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/masked-language-model-%E6%8E%A9%E7%A0%81%E8%AF%AD%E8%A8%80%E6%A8%A1%E5%9E%8B/">Masked Language Model (掩码语言模型)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/transformer-%E8%BD%AC%E6%8D%A2%E5%99%A8%E6%9E%B6%E6%9E%84/">Transformer (转换器架构)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/tokenization-%E5%88%86%E8%AF%8D/">Tokenization (分词)</a></li> </ul>ExBERThttps://terms-en.ai-term-hub.com/zh/terms/exbert/Sat, 18 Jul 2026 11:16:37 +0000https://terms-en.ai-term-hub.com/zh/terms/exbert/<h2 id="definition">Definition</h2> <p>ExBERT通过分析不同层中单个注意力头的重要性，为BERT Transformer模型提供可解释性。它使用基于梯度的归因或其他技术来量化每个组件对最终预测的贡献，从而帮助理解模型的决策过程。</p> <h3 id="summary">Summary</h3> <p>一种通过识别对特定输出贡献最大的注意力头和层来解释BERT预测的方法。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>注意力头</li> <li>模型可解释性</li> <li>梯度归因</li> <li>Transformer架构</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>调试NLP模型</li> <li>理解句法与语义处理</li> <li>医疗文本分析中的可解释AI</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/bert-bert%E6%A8%A1%E5%9E%8B/">bert (BERT模型)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/attention_mechanism-%E6%B3%A8%E6%84%8F%E5%8A%9B%E6%9C%BA%E5%88%B6/">attention_mechanism (注意力机制)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/explainable_ai-%E5%8F%AF%E8%A7%A3%E9%87%8A%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/">explainable_ai (可解释人工智能)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/transformer_models-transformer%E6%A8%A1%E5%9E%8B/">transformer_models (Transformer模型)</a></li> </ul>位置编码https://terms-en.ai-term-hub.com/zh/terms/positional_encoding/Sat, 18 Jul 2026 11:01:29 +0000https://terms-en.ai-term-hub.com/zh/terms/positional_encoding/<h2 id="definition">Definition</h2> <p>由于 Transformer 并行处理所有令牌，而不是像循环神经网络（RNN）那样按顺序处理，因此它缺乏对令牌顺序的固有认知。位置编码通过向输入嵌入添加特定的向量来保留这种顺序信息。</p> <h3 id="summary">Summary</h3> <p>一种将序列中令牌的相对或绝对位置信息注入 Transformer 模型的技术。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>序列顺序</li> <li>自注意力机制</li> <li>正弦函数</li> <li>令牌嵌入</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>机器翻译</li> <li>文本摘要</li> <li>语言建模</li> </ul> <h2 id="code-example">Code Example</h2> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#f92672">import</span> torch </span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> math </span></span><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">get_positional_encoding</span>(seq_len, d_model): </span></span><span style="display:flex;"><span> pe <span style="color:#f92672">=</span> torch<span style="color:#f92672">.</span>zeros(seq_len, d_model) </span></span><span style="display:flex;"><span> position <span style="color:#f92672">=</span> torch<span style="color:#f92672">.</span>arange(<span style="color:#ae81ff">0</span>, seq_len, dtype<span style="color:#f92672">=</span>torch<span style="color:#f92672">.</span>float)<span style="color:#f92672">.</span>unsqueeze(<span style="color:#ae81ff">1</span>) </span></span><span style="display:flex;"><span> div_term <span style="color:#f92672">=</span> torch<span style="color:#f92672">.</span>exp(torch<span style="color:#f92672">.</span>arange(<span style="color:#ae81ff">0</span>, d_model, <span style="color:#ae81ff">2</span>)<span style="color:#f92672">.</span>float() <span style="color:#f92672">*</span> (<span style="color:#f92672">-</span>math<span style="color:#f92672">.</span>log(<span style="color:#ae81ff">10000.0</span>) <span style="color:#f92672">/</span> d_model)) </span></span><span style="display:flex;"><span> pe[:, <span style="color:#ae81ff">0</span>::<span style="color:#ae81ff">2</span>] <span style="color:#f92672">=</span> torch<span style="color:#f92672">.</span>sin(position <span style="color:#f92672">*</span> div_term) </span></span><span style="display:flex;"><span> pe[:, <span style="color:#ae81ff">1</span>::<span style="color:#ae81ff">2</span>] <span style="color:#f92672">=</span> torch<span style="color:#f92672">.</span>cos(position <span style="color:#f92672">*</span> div_term) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> pe<span style="color:#f92672">.</span>unsqueeze(<span style="color:#ae81ff">0</span>) </span></span></code></pre></div><h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/transformer-%E6%9E%B6%E6%9E%84/">Transformer 架构</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E5%B5%8C%E5%85%A5-embedding/">嵌入 (Embedding)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E6%B3%A8%E6%84%8F%E5%8A%9B%E6%9C%BA%E5%88%B6-attention-mechanism/">注意力机制 (Attention Mechanism)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E6%97%8B%E8%BD%AC%E4%BD%8D%E7%BD%AE%E7%BC%96%E7%A0%81-rope/">旋转位置编码 (RoPE)</a></li> </ul>Encoderhttps://terms-en.ai-term-hub.com/zh/terms/encoder/Sat, 18 Jul 2026 10:59:51 +0000https://terms-en.ai-term-hub.com/zh/terms/encoder/<h2 id="definition">Definition</h2> <p>编码器处理原始输入序列或数据结构，并将它们转换为潜在空间表示，通常称为嵌入或代码。它们是 Transformer 和自编码器等架构的核心部分。</p> <h3 id="summary">Summary</h3> <p>编码器是神经网络的组件，它将输入数据转换为压缩的、有意义的表示形式。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>特征提取</li> <li>潜在空间</li> <li>序列处理</li> <li>压缩</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>在 Transformer 模型中处理输入文本</li> <li>在去噪自编码器中压缩图像</li> <li>从评论中提取情感特征</li> </ul> <h2 id="code-example">Code Example</h2> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#f92672">import</span> torch.nn <span style="color:#66d9ef">as</span> nn </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#66d9ef">class</span> <span style="color:#a6e22e">SimpleEncoder</span>(nn<span style="color:#f92672">.</span>Module): </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">def</span> __init__(self, input_dim, hidden_dim): </span></span><span style="display:flex;"><span> super()<span style="color:#f92672">.</span>__init__() </span></span><span style="display:flex;"><span> self<span style="color:#f92672">.</span>fc <span style="color:#f92672">=</span> nn<span style="color:#f92672">.</span>Linear(input_dim, hidden_dim) </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">def</span> <span style="color:#a6e22e">forward</span>(self, x): </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> torch<span style="color:#f92672">.</span>relu(self<span style="color:#f92672">.</span>fc(x)) </span></span></code></pre></div><h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/decoder-%E8%A7%A3%E7%A0%81%E5%99%A8/">Decoder (解码器)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/transformer-%E8%BD%AC%E6%8D%A2%E5%99%A8%E6%9E%B6%E6%9E%84/">Transformer (转换器架构)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/autoencoder-%E8%87%AA%E7%BC%96%E7%A0%81%E5%99%A8/">Autoencoder (自编码器)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/latent-variable-%E6%BD%9C%E5%8F%98%E9%87%8F/">Latent Variable (潜变量)</a></li> </ul>适配器https://terms-en.ai-term-hub.com/zh/terms/adapter/Sat, 18 Jul 2026 10:59:15 +0000https://terms-en.ai-term-hub.com/zh/terms/adapter/<h2 id="definition">Definition</h2> <p>适配器是一种参数高效的微调技术，主要用于大型语言模型和Transformer架构中。与计算成本高昂的更新所有模型权重不同，适配器仅训练少量新增参数，从而保留预训练知识并降低资源消耗。</p> <h3 id="summary">Summary</h3> <p>插入预训练模型中的轻量级模块，用于针对特定下游任务进行高效微调。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>参数高效微调</li> <li>迁移学习</li> <li>模块化架构</li> <li>灾难性遗忘</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>为客服聊天机器人微调大型语言模型</li> <li>适配视觉模型以进行医学图像分析</li> <li>高效部署多个领域特定的模型</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/lora-%E4%BD%8E%E7%A7%A9%E9%80%82%E5%BA%94/">LoRA (低秩适应)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/prompt-tuning-%E6%8F%90%E7%A4%BA%E5%BE%AE%E8%B0%83/">Prompt Tuning (提示微调)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/fine-tuning-%E5%BE%AE%E8%B0%83/">Fine-Tuning (微调)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/transformer-transformer%E6%9E%B6%E6%9E%84/">Transformer (Transformer架构)</a></li> </ul>注意力机制https://terms-en.ai-term-hub.com/zh/terms/attention/Sat, 18 Jul 2026 10:59:15 +0000https://terms-en.ai-term-hub.com/zh/terms/attention/<h2 id="definition">Definition</h2> <p>注意力机制使模型在处理输入（特别是文本等序列数据）时能够关注相关信息。通过计算注意力分数，模型确定哪些元素对当前任务最相关，从而捕捉长距离依赖关系并增强上下文理解。</p> <h3 id="summary">Summary</h3> <p>一种允许神经网络动态权衡输入序列不同部分重要性的机制。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>自注意力</li> <li>上下文加权</li> <li>长距离依赖</li> <li>Transformer架构</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>跨语言机器翻译</li> <li>长文档摘要</li> <li>图像描述生成和视觉问答</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/transformer-transformer%E6%9E%B6%E6%9E%84/">Transformer (Transformer架构)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/self-attention-%E8%87%AA%E6%B3%A8%E6%84%8F%E5%8A%9B/">Self-Attention (自注意力)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/multi-head-attention-%E5%A4%9A%E5%A4%B4%E6%B3%A8%E6%84%8F%E5%8A%9B/">Multi-Head Attention (多头注意力)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/sequence-modeling-%E5%BA%8F%E5%88%97%E5%BB%BA%E6%A8%A1/">Sequence Modeling (序列建模)</a></li> </ul>自注意力机制https://terms-en.ai-term-hub.com/zh/terms/self_attention/Sat, 18 Jul 2026 10:54:51 +0000https://terms-en.ai-term-hub.com/zh/terms/self_attention/<h2 id="definition">Definition</h2> <p>自注意力使模型能够同时捕捉序列中所有位置之间的依赖关系，无论距离远近。通过计算每对标记之间的注意力分数，它使得……</p> <h3 id="summary">Summary</h3> <p>一种允许神经网络根据彼此之间的相对重要性来权衡输入序列不同部分的机制。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>查询-键-值 (Query-Key-Value)</li> <li>注意力分数</li> <li>上下文加权</li> <li>并行处理</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>机器翻译</li> <li>文本摘要</li> <li>通过视觉Transformer进行图像分类</li> </ul> <h2 id="code-example">Code Example</h2> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#f92672">import</span> torch.nn <span style="color:#66d9ef">as</span> nn </span></span><span style="display:flex;"><span>attn <span style="color:#f92672">=</span> nn<span style="color:#f92672">.</span>MultiheadAttention(embed_dim<span style="color:#f92672">=</span><span style="color:#ae81ff">512</span>, num_heads<span style="color:#f92672">=</span><span style="color:#ae81ff">8</span>) </span></span></code></pre></div><h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/transformer-%E5%8F%98%E6%8D%A2%E5%99%A8/">Transformer (变换器)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/multi-head-attention-%E5%A4%9A%E5%A4%B4%E6%B3%A8%E6%84%8F%E5%8A%9B/">Multi-Head Attention (多头注意力)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/embeddings-%E5%B5%8C%E5%85%A5%E5%90%91%E9%87%8F/">Embeddings (嵌入向量)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/sequence-modeling-%E5%BA%8F%E5%88%97%E5%BB%BA%E6%A8%A1/">Sequence Modeling (序列建模)</a></li> </ul>