A reference from Weave

LLM fundamentals glossary

Build a practical understanding of language models, prompts, tokens, and context. Focus on the concepts that affect real applications and engineering workflows.

All terms

148 terms

  • Absolute positional embedding

    Absolute positional embedding is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Activation

    Activation is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Activation patching

    Activation patching is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Approximate nearest neighbor

    Approximate nearest neighbor is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Attention head

    Attention head is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Automatic prompt optimization

    Automatic prompt optimization is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Beam search decoding

    Beam search decoding is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Bidirectional encoder

    Bidirectional encoder is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • BOS token

    BOS token is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Catastrophic forgetting

    Catastrophic forgetting is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Causal language modeling

    Causal language modeling is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Causal mask

    Causal mask is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Chain-of-thought prompting

    Chain-of-thought prompting is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Checkpoint

    Checkpoint is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Circuit

    Circuit is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Code representation

    Code representation is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Constrained decoding

    Constrained decoding is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Context length

    Context length is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Context packing

    Context packing is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Context ranking

    Context ranking is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Context selection

    Context selection is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Context truncation

    Context truncation is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Context window

    A context window is the amount of information a language model can consider within a request and its generation process, usually expressed in tokens. The applicable limits and accounting rules depend on the model and serving interface.

    LLM fundamentals

  • Context window utilization

    Context window utilization is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Contextual instruction

    Contextual instruction is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Continued pretraining

    Continued pretraining is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Contrastive decoding

    Contrastive decoding is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Contrastive learning

    Contrastive learning is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Conversation history

    Conversation history is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Cosine similarity

    Cosine similarity is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Cross-attention

    Cross-attention is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Curriculum learning

    Curriculum learning is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Data deduplication

    Data deduplication is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Data filtering

    Data filtering is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Data mixture

    Data mixture is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Data mixture weighting

    Data mixture weighting is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Decoder-only model

    Decoder-only model is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Delimiting

    Delimiting is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Denoising autoencoding

    Denoising autoencoding is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Document representation

    Document representation is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Domain adaptation

    Domain adaptation is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Early stopping

    Early stopping is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Early stopping generation

    Early stopping generation is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Embedding pooling

    Embedding pooling is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Embedding space

    Embedding space is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Embeddings

    An embedding is a numeric representation of content produced by a model so that items with related properties can be compared in a vector space. Embeddings are commonly used for semantic search, retrieval, clustering, and recommendation.

    LLM fundamentals

  • Encoder-decoder model

    Encoder-decoder model is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • EOS token

    EOS token is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Epoch

    Epoch is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Euclidean distance

    Euclidean distance is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals