https://terms-en.ai-term-hub.com/zh/tags/data-preprocessing/Recent content in Data Preprocessing on 中文AI术语词典Hugozh-cnSat, 18 Jul 2026 11:44:45 +0000https://terms-en.ai-term-hub.com/zh/terms/normalization/Sat, 18 Jul 2026 11:28:24 +0000https://terms-en.ai-term-hub.com/zh/terms/normalization/<h2 id="definition">Definition</h2> <p>常见方法包括最小-最大缩放和Z分数标准化。此过程确保具有较大量级的特征不会主导学习算法，特别是在基于梯度的优化中。</p> <h3 id="summary">Summary</h3> <p>归一化是一种数据预处理技术，将数值特征缩放到标准范围（通常为0到1之间），以改善模型的收敛速度和性能。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>最小-最大缩放</li> <li>Z分数标准化</li> <li>特征缩放</li> <li>梯度下降稳定性</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>预处理图像像素值</li> <li>为神经网络准备表格数据</li> <li>提高回归模型的准确性</li> </ul> <h2 id="code-example">Code Example</h2> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#f92672">from</span> sklearn.preprocessing <span style="color:#f92672">import</span> MinMaxScaler </span></span><span style="display:flex;"><span><span style="color:#f92672">import</span> numpy <span style="color:#66d9ef">as</span> np </span></span><span style="display:flex;"><span>data <span style="color:#f92672">=</span> np<span style="color:#f92672">.</span>array([[<span style="color:#ae81ff">10</span>], [<span style="color:#ae81ff">20</span>], [<span style="color:#ae81ff">30</span>]]) </span></span><span style="display:flex;"><span>scaler <span style="color:#f92672">=</span> MinMaxScaler() </span></span><span style="display:flex;"><span>normalized_data <span style="color:#f92672">=</span> scaler<span style="color:#f92672">.</span>fit_transform(data) </span></span></code></pre></div><h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/standardization-%E6%A0%87%E5%87%86%E5%8C%96/">Standardization (标准化)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/data-preprocessing-%E6%95%B0%E6%8D%AE%E9%A2%84%E5%A4%84%E7%90%86/">Data Preprocessing (数据预处理)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/feature-engineering-%E7%89%B9%E5%BE%81%E5%B7%A5%E7%A8%8B/">Feature Engineering (特征工程)</a></li> </ul>https://terms-en.ai-term-hub.com/zh/terms/label_noise/Sat, 18 Jul 2026 11:23:29 +0000https://terms-en.ai-term-hub.com/zh/terms/label_noise/<h2 id="definition">Definition</h2> <p>标签噪声指的是数据实例的真实类别标签与训练数据集中提供的标签之间的差异。这可能源于人工标注错误、模糊的数据点或数据采集过程中的缺陷。标签噪声会降低模型的泛化能力和准确性，因此研究鲁棒学习算法以减轻其影响至关重要。</p> <h3 id="summary">Summary</h3> <p>用于监督机器学习训练的数据集中，目标标签存在的错误或不一致性。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>数据质量</li> <li>鲁棒学习</li> <li>标注错误</li> <li>对称/非对称噪声</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>在众包数据上训练模型</li> <li>处理现实世界中不完美的数据集</li> <li>提高模型鲁棒性</li> </ul> <h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/data_cleaning-%E6%95%B0%E6%8D%AE%E6%B8%85%E6%B4%97/">data_cleaning (数据清洗)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/robust_statistics-%E9%B2%81%E6%A3%92%E7%BB%9F%E8%AE%A1%E5%AD%A6/">robust_statistics (鲁棒统计学)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/supervised_learning-%E7%9B%91%E7%9D%A3%E5%AD%A6%E4%B9%A0/">supervised_learning (监督学习)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/outlier_detection-%E5%BC%82%E5%B8%B8%E5%80%BC%E6%A3%80%E6%B5%8B/">outlier_detection (异常值检测)</a></li> </ul>https://terms-en.ai-term-hub.com/zh/terms/bpe/Sat, 18 Jul 2026 10:59:27 +0000https://terms-en.ai-term-hub.com/zh/terms/bpe/<h2 id="definition">Definition</h2> <p>字节对编码（BPE）是一种数据压缩技术，经过调整后应用于自然语言处理中，以处理未登录词（Out-of-Vocabulary）。它从单个字符的词汇表开始，并迭代地合并最频繁出现的字符对，直到达到预定的词汇表大小或收敛。这种方法允许模型将罕见词分解为更常见的子词单元，从而提高对未知词汇的处理能力。</p> <h3 id="summary">Summary</h3> <p>字节对编码是一种用于子词分词的算法，它通过迭代合并出现频率最高的字符对来构建词汇表。</p> <h2 id="key-concepts">Key Concepts</h2> <ul> <li>子词分词</li> <li>词汇表合并</li> <li>频率分析</li> <li>未登录词处理</li> </ul> <h2 id="use-cases">Use Cases</h2> <ul> <li>为大语言模型预处理文本</li> <li>处理形态丰富的语言</li> <li>减少神经网络中的词汇表规模</li> </ul> <h2 id="code-example">Code Example</h2> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span><span style="color:#f92672">import</span> tiktoken </span></span><span style="display:flex;"><span>enc <span style="color:#f92672">=</span> tiktoken<span style="color:#f92672">.</span>get_encoding(<span style="color:#e6db74">&#34;cl100k_base&#34;</span>) </span></span><span style="display:flex;"><span>tokens <span style="color:#f92672">=</span> enc<span style="color:#f92672">.</span>encode(<span style="color:#e6db74">&#34;unhappiness&#34;</span>) </span></span><span style="display:flex;"><span>print(tokens) </span></span></code></pre></div><h2 id="related-terms">Related Terms</h2> <ul> <li><a href="https://terms-en.ai-term-hub.com/en/terms/wordpiece-wordpiece%E5%88%86%E8%AF%8D%E7%AE%97%E6%B3%95/">WordPiece (WordPiece分词算法)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/sentencepiece-sentencepiece%E5%88%86%E8%AF%8D%E5%B7%A5%E5%85%B7%E5%BA%93/">SentencePiece (SentencePiece分词工具库)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/tokenization-%E5%88%86%E8%AF%8D/">Tokenization (分词)</a></li> <li><a href="https://terms-en.ai-term-hub.com/en/terms/subword-units-%E5%AD%90%E8%AF%8D%E5%8D%95%E5%85%83/">Subword Units (子词单元)</a></li> </ul>