Concept · embedding

Embedding

A list of numbers representing a token's or document's meaning as a position in space.

◷ Share view, as of June 2026: confidence, rationales and priors are hidden. The questions, stances, claims, tests and evidence stay visible.

Settled

Broad technical consensus on what this term means. Disagreements that invoke it are about the world, not the word.

What it is

An embedding turns a discrete symbol, a token, a sentence, a whole document, into a vector of numbers placed so that similar meanings sit near each other. "Dog" lands close to "puppy" and far from "invoice." This is the representation models actually compute with; text goes in, geometry happens, text comes out.

Embeddings also stand alone as infrastructure: semantic search, recommendation, clustering, and the retrieval step in RAG all work by comparing embedding distances rather than matching keywords.

Understand first

tokenToken · The basic unit of text a language model reads and writes, roughly a word fragment.

Builds toward

attention-mechanismAttention mechanism · The operation that lets a model weigh every token against every other to decide what matters in context.positional-encodingPositional encoding · Information about token order, added to the embeddings because attention alone is order-blind.ragRAG (retrieval-augmented generation) · Pairing a model with a search step that retrieves relevant documents into its context before it answers.residual-streamResidual stream · The running representation a transformer carries from layer to layer, which each block adds to rather than replaces.transformerTransformer · The neural-network architecture, built around attention, that underlies virtually all modern AI models.