Polysemanticity

Sat, 18 Jul 2026 10:10:59 +0000

Definition

Polysemanticity is a characteristic observed in deep neural networks, particularly in transformers, where a single neuron may activate in response to several unrelated or semantically distinct features. This contrasts with monosemantic neurons, which respond to only one specific concept. Understanding polysemanticity is crucial for interpretability research, as it complicates efforts to map specific network components to human-understandable concepts, necessitating advanced techniques like sparse autoencoders for disentanglement.

Summary

The phenomenon where individual neurons in neural networks respond to multiple distinct concepts.

Neurons on English AI Terms Dictionary

Polysemanticity

Definition

Summary