Machine Learning on English AI Terms Dictionary

Underfitting

Sat, 18 Jul 2026 10:19:06 +0000

Definition

Underfitting occurs when a statistical model or machine learning algorithm cannot approximate the function mapping inputs to outputs accurately. This usually happens when the model is too simple for the complexity of the data, such as using linear regression on non-linear data. It results in poor performance on both training and test datasets. To resolve underfitting, practitioners may increase model complexity, add more relevant features, reduce regularization, or train the model for more epochs until it learns the patterns effectively.

Offline learning

Sat, 18 Jul 2026 10:09:37 +0000

Definition

Also known as batch learning, offline learning involves training machine learning models on a fixed dataset collected previously. Unlike online learning, the model does not update its parameters in real-time as new data arrives. This approach is computationally efficient for large-scale training but requires periodic retraining to incorporate new information, making it suitable for scenarios where immediate adaptation is not critical.

Summary

Offline learning is a training paradigm where models are trained on static datasets without interacting with the live environment during the learning phase.

Leave-one-out cross-validation

Sat, 18 Jul 2026 10:04:58 +0000

Definition

Leave-one-out cross-validation (LOOCV) is a specific case of k-fold cross-validation where k equals the number of samples in the dataset. It provides a nearly unbiased estimate of model performance because each observation serves as the test set exactly once. While computationally expensive due to the need to train the model n times, it is highly effective for small datasets where maximizing training data usage is critical for robust evaluation.

Limited Memory AI

Sat, 18 Jul 2026 10:04:58 +0000

Definition

Limited Memory AI represents the second level of AI capability, where systems can learn from historical data and adjust their behavior accordingly. Unlike reactive machines, these systems retain information about previous interactions or datasets to improve performance over time. This category encompasses supervised learning, reinforcement learning, and neural networks, enabling applications like recommendation engines, image recognition, and autonomous driving systems.

Summary

AI systems capable of storing past data and experiences to inform future decisions, forming the basis of most modern machine learning applications.

Lazy learning

Sat, 18 Jul 2026 10:04:23 +0000

Definition

Lazy learners, such as k-Nearest Neighbors (k-NN), memorize the entire training dataset and perform computations only when making predictions. This contrasts with eager learning, which builds a generalized model upfront. While lazy learning can adapt quickly to new data without retraining, it suffers from high computational costs during inference and large memory requirements due to storing all training examples.

Summary

A learning approach that delays generalization until classification time, storing training instances rather than building an explicit model.

Dataset shift

Sat, 18 Jul 2026 09:53:01 +0000

Definition

Dataset shift occurs when the distribution of data used to train a machine learning model differs from the distribution of data encountered during inference. This discrepancy can lead to significant performance degradation. Common types include covariate shift, prior probability shift, and concept drift. Addressing dataset shift is critical for ensuring model robustness and generalization in real-world applications, often requiring techniques like domain adaptation or continuous monitoring.

Summary

Dataset shift refers to the phenomenon where the statistical properties of the input data change between training and deployment.

Data Augmentation

Sat, 18 Jul 2026 09:52:41 +0000

Definition

This method artificially expands the training dataset by creating modified versions of existing samples, such as rotating images, adding noise to audio, or synonym replacement in text. It helps prevent overfitting by exposing the model to a wider variety of scenarios during training, thereby improving generalization performance. It is particularly crucial in domains where collecting large amounts of real-world labeled data is expensive or difficult.

Summary

Data augmentation is a technique used to increase the diversity and size of training datasets by applying transformations to existing data points.

Cross-validation

Sat, 18 Jul 2026 09:52:18 +0000

Definition

Cross-validation is a statistical method used to estimate the skill of machine learning models. The most common form is k-fold cross-validation, where the data is split into k equal parts. The model is trained on k-1 folds and validated on the remaining fold, repeating this process k times so each fold serves as the validation set once. This approach provides a more robust estimate of model performance than a single train-test split, helping to detect overfitting and ensuring the model generalizes well to unseen data.

Concept Drift

Sat, 18 Jul 2026 09:51:20 +0000

Definition

Concept drift is a phenomenon in machine learning where the relationship between input features and the target output changes as new data arrives. This often happens in dynamic environments where user behavior or underlying physical processes evolve. If a model is not updated or adapted to these changes, its predictive accuracy will decline. Detecting and handling concept drift is essential for maintaining robust performance in production systems, requiring techniques like retraining or online learning.

Ball tree

Sat, 18 Jul 2026 09:47:37 +0000

Definition

A Ball tree partitions data points into nested hyperspheres (balls) rather than hyperrectangles. This structure allows for efficient pruning during nearest neighbor queries by calculating distances between balls rather than individual points. It is particularly advantageous in high-dimensional spaces where other structures like KD-trees may suffer from the curse of dimensionality, providing faster search times for k-NN algorithms.

Summary

A binary tree data structure used to organize points in space, optimizing nearest neighbor searches in high-dimensional datasets.

Anomaly detection

Sat, 18 Jul 2026 09:45:36 +0000

Definition

Anomaly detection, also known as outlier detection, involves analyzing data to find patterns that do not conform to expected behavior. It is widely used in cybersecurity, fraud detection, and system monitoring to identify potential threats or errors. Techniques range from statistical methods to machine learning models like isolation forests and autoencoders, which learn normal behavior and flag deviations as anomalies.

Summary

The process of identifying rare items, events, or observations that deviate significantly from the majority of the data.

Vector Database

Sat, 18 Jul 2026 09:37:52 +0000

Definition

Vector databases optimize the storage and retrieval of unstructured data by converting it into numerical embeddings. They use algorithms like Approximate Nearest Neighbor (ANN) to efficiently find similar items based on distance metrics. This technology is critical for AI applications requiring semantic search, recommendation engines, and similarity matching, enabling fast retrieval from massive datasets where traditional relational databases fail.

Summary

A specialized database designed to store, index, and query high-dimensional vectors representing data features.

Reinforcement

Sat, 18 Jul 2026 09:36:45 +0000

Definition

Reinforcement is a fundamental psychological and computational mechanism where an agent’s actions are shaped by consequences. In machine learning, it involves providing positive feedback (rewards) for desirable outcomes and negative feedback (penalties) for undesirable ones. This feedback loop allows systems to learn optimal strategies over time without explicit supervision, focusing on maximizing cumulative long-term reward rather than immediate accuracy.

Summary

Reinforcement refers to the process of modifying behavior through rewards or punishments to optimize decision-making.

Reinforcement Learning

Sat, 18 Jul 2026 09:36:45 +0000

Definition

Reinforcement Learning (RL) is a branch of machine learning focused on how intelligent agents ought to take actions in an environment to maximize the notion of cumulative reward. Unlike supervised learning, RL does not need labeled input/output pairs but instead focuses on finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge). The agent learns a policy that maps states to actions, improving over time through trial and error.