Generative Models on English AI Terms Dictionary

Product of experts

Sat, 18 Jul 2026 10:11:46 +0000

Definition

The Product of Experts (PoE) is a method for constructing complex probability distributions by combining simpler ones. Unlike the ‘Mixture of Experts,’ which averages probabilities, PoE multiplies them, resulting in a distribution that is zero wherever any single expert assigns zero probability. This creates a more peaked and constrained distribution, effectively requiring all experts to agree on a valid configuration. It is particularly useful in energy-based models and deep learning architectures for capturing intricate dependencies in data, such as image textures or natural language structures.

Inception Score

Sat, 18 Jul 2026 10:02:49 +0000

Definition

The Inception Score (IS) is a statistical measure introduced to assess the performance of Generative Adversarial Networks (GANs) and other generative models. It combines two factors: image quality (clarity) and variety (diversity). A higher score indicates that the generated images are sharp and distinct from one another. While popular, it has limitations as it does not compare generated images to real ones directly, potentially allowing low-quality but diverse outputs to score well.

Energy-based model

Sat, 18 Jul 2026 09:56:53 +0000

Definition

Energy-Based Models (EBMs) define a probability distribution over input data using an unnormalized density function derived from an energy function. The energy function maps data points to real numbers, where lower energies correspond to higher probabilities. EBMs are flexible and can model complex multimodal distributions but often require computationally intensive sampling methods, such as Markov Chain Monte Carlo, for inference and training compared to normalized models like softmax classifiers.

Diffusion

Sat, 18 Jul 2026 09:31:18 +0000

Definition

Diffusion models are a class of generative AI that learn to reverse a stochastic process of adding noise to data. By training a neural network to predict and remove this noise step-by-step, they can generate high-quality, diverse samples such as images, audio, or text. These models have become state-of-the-art in creative tasks due to their stability and ability to produce realistic outputs compared to earlier GANs.

Summary

A generative modeling technique that creates data by reversing a gradual noising process to reconstruct clean samples.