Probabilistic Models on English AI Terms Dictionary

Reparameterization trick

Sat, 18 Jul 2026 10:14:07 +0000

Definition

The reparameterization trick is a fundamental method used in variational autoencoders and other probabilistic models. It allows gradients to flow through stochastic nodes by expressing a random variable z as a differentiable function of distribution parameters and an independent noise variable epsilon. This enables the use of backpropagation to optimize the expected log-likelihood, making training of latent variable models efficient and stable via Monte Carlo estimation.

Summary

A technique that separates stochastic variables from learnable parameters to enable gradient-based optimization in variational inference.

Expectation propagation

Sat, 18 Jul 2026 09:57:25 +0000

Definition

Expectation Propagation (EP) approximates intractable integrals by iteratively refining Gaussian approximations to the true posterior distribution. It minimizes the Kullback-Leibler divergence between the approximate and true distributions by matching moments. EP is widely used in Bayesian machine learning for tasks like classification and regression where exact inference is computationally prohibitive, offering a balance between accuracy and efficiency.

Summary

An approximate inference algorithm used to estimate posterior distributions in complex probabilistic graphical models.

Bayesian interpretation of kernel regularization

Sat, 18 Jul 2026 09:47:51 +0000

Definition

This concept establishes that minimizing a regularized risk functional with a specific kernel is equivalent to finding the maximum a posteriori (MAP) estimate in a Bayesian framework. Specifically, it interprets the regularization term as a log-prior over functions, often corresponding to a Gaussian Process prior. This connection allows practitioners to apply Bayesian uncertainty quantification techniques to deterministic kernel methods, providing probabilistic predictions and insights into model confidence.

Summary

A theoretical framework linking kernel methods like SVMs to Gaussian Processes under a Bayesian prior assumption.