Model Training on English AI Terms Dictionary

Underfitting

Sat, 18 Jul 2026 10:19:06 +0000

Definition

Underfitting occurs when a statistical model or machine learning algorithm cannot approximate the function mapping inputs to outputs accurately. This usually happens when the model is too simple for the complexity of the data, such as using linear regression on non-linear data. It results in poor performance on both training and test datasets. To resolve underfitting, practitioners may increase model complexity, add more relevant features, reduce regularization, or train the model for more epochs until it learns the patterns effectively.

Optimization

Sat, 18 Jul 2026 09:41:54 +0000

Definition

In machine learning, optimization refers to the algorithms used to adjust model parameters to minimize a loss function, thereby improving model performance. Common methods include Gradient Descent and its variants like Adam or SGD. The goal is to navigate the parameter space efficiently to find global or local minima, ensuring the model generalizes well to unseen data by reducing the discrepancy between predicted and actual outputs.

Summary

The mathematical process of minimizing or maximizing an objective function to find the best solution parameters.

Dropout

Sat, 18 Jul 2026 09:40:59 +0000

Definition

In neural networks, dropout prevents overfitting by temporarily removing a random subset of neurons during each training step. This forces the network to learn robust features that are useful in conjunction with many other random subsets of neurons, rather than relying on specific local patterns. During inference, all neurons are used, but their outputs are scaled to account for the increased activity compared to training time.

Summary

Dropout is a regularization technique that randomly ignores neurons during training to prevent overfitting.