Generalization on English AI Terms Dictionary

Statistical learning theory

Sat, 18 Jul 2026 10:16:56 +0000

Definition

Statistical learning theory (SLT) is a branch of statistics and computer science that studies how specific algorithms can generalize from finite training samples to unseen data. It focuses on bounding the error between empirical performance on training data and true expected risk. Key components include VC dimension and Rademacher complexity, which measure model capacity. SLT helps determine sample complexity requirements and ensures that models do not merely memorize noise but learn underlying patterns, providing guarantees for convergence and stability in supervised learning settings.

Rademacher complexity

Sat, 18 Jul 2026 10:13:36 +0000

Definition

Rademacher complexity evaluates how well a hypothesis class can correlate with random labels (noise). It serves as a proxy for the model’s capacity or flexibility. Lower complexity suggests better generalization, meaning the model is less likely to overfit training data. It is fundamental in deriving generalization bounds for supervised learning algorithms, helping practitioners understand the trade-off between model complexity and empirical performance.

Summary

A statistical measure used in learning theory to quantify the richness of a function class by its ability to fit random noise.

Domain Adaptation

Sat, 18 Jul 2026 09:56:08 +0000

Definition

Domain adaptation addresses the challenge when training and testing data come from different distributions. By aligning feature representations between a labeled source domain and an unlabeled or sparsely labeled target domain, models can generalize better to new environments. This technique is crucial for deploying AI systems in real-world scenarios where data characteristics shift over time or vary across regions, ensuring robustness without requiring extensive new labeled datasets.

Summary

A machine learning method that improves model performance on a target domain by leveraging knowledge from a source domain.

Double Descent

Sat, 18 Jul 2026 09:56:08 +0000

Definition

Double descent challenges the traditional bias-variance tradeoff by showing that highly overparameterized models can achieve low test error despite interpolating training data. Initially, error rises as models memorize noise, but further increasing capacity allows the model to find smoother solutions that generalize well. This behavior is particularly observed in deep neural networks, explaining why larger models often perform better than smaller ones even when they fit training data perfectly.

Zero-shot Learning

Sat, 18 Jul 2026 09:43:55 +0000

Definition

Zero-shot learning enables a machine learning model to classify instances of classes that were not present in its training dataset. Instead of relying on labeled examples for every possible class, the model uses auxiliary information, such as textual descriptions or attribute vectors, to infer relationships between known and unknown classes. This approach significantly reduces the need for extensive labeled data and allows models to generalize to new concepts based on learned semantic structures.

Overfitting

Sat, 18 Jul 2026 09:41:54 +0000

Definition

Overfitting occurs when a model learns the training data too well, including its random noise and outliers, resulting in excellent performance on training data but poor performance on new, unseen test data. This happens because the model becomes overly complex relative to the amount of training data available. Techniques like regularization, dropout, early stopping, and cross-validation are commonly employed to mitigate overfitting and improve the model’s ability to generalize.

Domain

Sat, 18 Jul 2026 09:31:32 +0000

Definition

In machine learning, particularly in transfer learning, a domain is defined by two components: the feature space (the set of all possible inputs) and the marginal probability distribution of those inputs. For example, images taken in daylight and images taken at night constitute different domains due to their distinct distributions, even if they share the same feature space (pixels). Understanding domains is critical for addressing domain shift, where a model trained on one domain performs poorly on another, necessitating techniques like domain adaptation to bridge the gap between source and target distributions.