Feature Engineering on English AI Terms Dictionary

Random feature

Sat, 18 Jul 2026 10:13:36 +0000

Definition

Random feature maps transform inputs into a new space where linear models can approximate non-linear kernel functions. This approach, often associated with the Nystrom method or Fourier features, allows for scalable kernel regression and classification. By avoiding the explicit computation of large kernel matrices, it reduces computational complexity from quadratic to linear in the number of samples, making it suitable for large-scale datasets.

Summary

A technique that maps input data into a higher-dimensional space using random projections to approximate kernel methods efficiently.

Bag-of-words model

Sat, 18 Jul 2026 09:47:37 +0000

Definition

This natural language processing technique represents text as a multiset of words, disregarding syntax and sequence. It converts documents into numerical vectors based on word frequency or presence. While it loses contextual information like word order, it remains computationally efficient and effective for tasks such as text classification, spam detection, and topic modeling. It serves as a foundational feature extraction method before more advanced embeddings like Word2Vec became prevalent.

Encoder

Sat, 18 Jul 2026 09:40:59 +0000

Definition

Encoders process raw input sequences or data structures and convert them into latent space representations, often called embeddings or codes. They are central to architectures like Transformers and Autoencoders. The encoder’s goal is to capture essential features and contextual information while discarding noise, creating a compact summary that downstream components, such as decoders or classifiers, can utilize effectively for prediction or generation tasks.

Summary

An encoder is a component of a neural network that transforms input data into a compressed, meaningful representation.