https://terms-en.ai-term-hub.com/zh/tags/compression/Recent content in Compression on 中文AI术语词典Hugozh-cnSat, 18 Jul 2026 11:44:45 +0000https://terms-en.ai-term-hub.com/zh/terms/pruning/Sat, 18 Jul 2026 11:30:52 +0000https://terms-en.ai-term-hub.com/zh/terms/pruning/<h2 id="definition">Definition</h2> <p>剪枝涉及识别并消除神经网络中对输出准确率贡献最小的神经元、连接或滤波器。通过移除这些冗余元素，模型变得更加紧凑，从而降低存储需求和计算开销，同时尽量保持原有的模型性能。</p> <h3 id="summary">Summary</h3> <p>一种模型压缩技术，通过移除冗余或重要性较低的参数来减小模型体积并提高推理速度。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>模型压缩</li> <li>冗余移除</li> <li>推理加速</li> <li>稀疏性</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>移动端AI部署</li> <li>边缘计算优化</li> <li>降低云端推理成本</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E9%87%8F%E5%8C%96-quantization/">量化 (Quantization)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E7%9F%A5%E8%AF%86%E8%92%B8%E9%A6%8F-knowledge-distillation/">知识蒸馏 (Knowledge Distillation)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E6%A8%A1%E5%9E%8B%E5%8E%8B%E7%BC%A9-model-compression/">模型压缩 (Model Compression)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E7%A8%80%E7%96%8F%E7%BD%91%E7%BB%9C-sparse-networks/">稀疏网络 (Sparse Networks)</a></li> </ul>https://terms-en.ai-term-hub.com/zh/terms/knowledge_distillation/Sat, 18 Jul 2026 11:23:05 +0000https://terms-en.ai-term-hub.com/zh/terms/knowledge_distillation/<h2 id="definition">Definition</h2> <p>知识蒸馏是一种机器学习方法，用于将庞大复杂的神经网络（教师模型）压缩为更小、更高效的网络（学生模型）。学生模型经过训练以模仿教师模型的行为。</p> <h3 id="summary">Summary</h3> <p>知识蒸馏是一种模型压缩技术，其中较小的学生模型通过学习模仿较大的教师模型的行为来工作。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>师生模型</li> <li>模型压缩</li> <li>软标签</li> <li>效率</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>在边缘设备上部署模型</li> <li>降低推理延迟</li> <li>降低云计算成本</li> </ul> <h2 id="code-example">Code Example</h2> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#f92672">import</span> torch </span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> torch.nn <span style="color:#66d9ef">as</span> nn </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#66d9ef">def</span> <span style="color:#a6e22e">distillation_loss</span>(student_logits, teacher_logits, temperature<span style="color:#f92672">=</span><span style="color:#ae81ff">2.0</span>): </span></span><span style="display:flex;"><span> T <span style="color:#f92672">=</span> temperature </span></span><span style="display:flex;"><span> student_probs <span style="color:#f92672">=</span> nn<span style="color:#f92672">.</span>functional<span style="color:#f92672">.</span>softmax(student_logits <span style="color:#f92672">/</span> T, dim<span style="color:#f92672">=</span><span style="color:#ae81ff">1</span>) </span></span><span style="display:flex;"><span> teacher_probs <span style="color:#f92672">=</span> nn<span style="color:#f92672">.</span>functional<span style="color:#f92672">.</span>softmax(teacher_logits <span style="color:#f92672">/</span> T, dim<span style="color:#f92672">=</span><span style="color:#ae81ff">1</span>) </span></span><span style="display:flex;"><span> <span style="color:#66d9ef">return</span> nn<span style="color:#f92672">.</span>functional<span style="color:#f92672">.</span>kl_div( </span></span><span style="display:flex;"><span> nn<span style="color:#f92672">.</span>functional<span style="color:#f92672">.</span>log_softmax(student_logits <span style="color:#f92672">/</span> T, dim<span style="color:#f92672">=</span><span style="color:#ae81ff">1</span>), </span></span><span style="display:flex;"><span> teacher_probs, </span></span><span style="display:flex;"><span> reduction<span style="color:#f92672">=</span><span style="color:#e6db74">&#39;batchmean&#39;</span> </span></span><span style="display:flex;"><span> ) <span style="color:#f92672">*</span> (T <span style="color:#f92672">*</span> T) </span></span></code></pre></div><h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/model-compression-%E6%A8%A1%E5%9E%8B%E5%8E%8B%E7%BC%A9/">Model Compression (模型压缩)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/pruning-%E5%89%AA%E6%9E%9D/">Pruning (剪枝)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/quantization-%E9%87%8F%E5%8C%96/">Quantization (量化)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/neural-networks-%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C/">Neural Networks (神经网络)</a></li> </ul>https://terms-en.ai-term-hub.com/zh/terms/distillation/Sat, 18 Jul 2026 10:50:42 +0000https://terms-en.ai-term-hub.com/zh/terms/distillation/<h2 id="definition">Definition</h2> <p>这一过程涉及将知识从复杂的高性能“教师”神经网络转移到更简单、高效的“学生”网络中。学生不仅从硬标签中学习，还从软标签（即教师模型的输出概率分布）中学习，从而捕捉更丰富的信息。</p> <h3 id="summary">Summary</h3> <p>知识蒸馏是一种模型压缩技术，其中较小的学生模型通过学习模仿较大的教师模型的行为来工作。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>师生架构</li> <li>软目标</li> <li>模型压缩</li> <li>推理效率</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>在移动设备上部署大型语言模型</li> <li>降低实时计算机视觉应用中的延迟</li> <li>优化边缘计算环境中的深度学习模型</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E9%87%8F%E5%8C%96-quantization/">量化 (Quantization)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E5%89%AA%E6%9E%9D-pruning/">剪枝 (Pruning)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E8%BF%81%E7%A7%BB%E5%AD%A6%E4%B9%A0-transfer-learning/">迁移学习 (Transfer Learning)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/%E7%A5%9E%E7%BB%8F%E6%9E%B6%E6%9E%84%E6%90%9C%E7%B4%A2-neural-architecture-search/">神经架构搜索 (Neural Architecture Search)</a></li> </ul>