LLM fundamentals

Embeddings

Also known as Vector embeddings, Text embeddings

By WeavePublished 1 min read

Definition

An embedding is a numeric representation of content produced by a model so that items with related properties can be compared in a vector space. Embeddings are commonly used for semantic search, retrieval, clustering, and recommendation.

Similarity in a vector space

An embedding model maps content into a list of numbers. A distance or similarity function then compares those lists. The result is useful because related content can be close even when it does not share the same words.

An illustrative search for a deployment regression might retrieve a discussion that says rollback rather than using the exact phrase in the question. That semantic relationship can improve discovery.

The model and the data both matter

An embedding model trained for general language may handle prose differently from source code, stack traces, or ticket identifiers. Chunk size, metadata, and the way content is updated also shape retrieval quality.

Evaluate which relevant records appear in the top results and whether unrelated records are mixed in. Check access controls before indexing private engineering data. A mathematically close record is not necessarily appropriate to show to the person asking.

Embeddings support retrieval. They do not by themselves prove that a generated answer is correct. Pair retrieval tests with answer evaluation and source inspection.

How this relates to Weave

Embeddings can help an engineering assistant find related code, incidents, or discussions before answering a question. Weave products should be evaluated on whether the retrieved records are relevant and permitted, not just whether vector similarity returns a result.

Explore Wooly

Sources and further reading

  1. Embeddings, OpenAI platform documentation