A reference from Weave

LLM fundamentals glossary

Build a practical understanding of language models, prompts, tokens, and context. Focus on the concepts that affect real applications and engineering workflows.

All terms

148 terms

  • Feature representation

    Feature representation is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Feed-forward network

    Feed-forward network is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Few-shot prompting

    Few-shot prompting is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Generalization

    Generalization is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Gradient accumulation

    Gradient accumulation is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Gradient clipping

    Gradient clipping is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Grammar-constrained generation

    Grammar-constrained generation is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Graph of thoughts

    Graph of thoughts is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Grouped-query attention

    Grouped-query attention is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Hallucination

    An AI hallucination is a generated statement or artifact that is unsupported, fabricated, or incorrect for the task and available evidence. A fluent answer can still contain hallucinations.

    LLM fundamentals

  • Hidden state

    Hidden state is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • In-context learning

    In-context learning is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Induction head

    Induction head is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Input context

    Input context is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Instruction following

    Instruction following is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Instruction hierarchy

    Instruction hierarchy is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Instruction hierarchy conflict

    Instruction hierarchy conflict is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Jailbreak

    Jailbreak is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Language-model objective

    Language-model objective is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Latent representation

    Latent representation is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Layer normalization

    Layer normalization is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Learning rate schedule

    Learning rate schedule is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Length penalty

    Length penalty is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • LLM inference

    LLM inference is the process of running a trained language model on an input to produce an output. For a text-generating model, it typically involves processing the input context and generating additional tokens according to a decoding strategy.

    LLM fundamentals

  • Logit lens

    Logit lens is a language-model concept about mechanism and information flow. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Masked language modeling

    Masked language modeling is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Masked span corruption

    Masked span corruption is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Max pooling

    Max pooling is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Maximum generation length

    Maximum generation length is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Mean pooling

    Mean pooling is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Mechanistic interpretability

    Mechanistic interpretability is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Minimum generation length

    Minimum generation length is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Mixed precision training

    Mixed precision training is a language-model concept about serving behavior and operational tradeoffs. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Multi-query attention

    Multi-query attention is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Nearest-neighbor search

    Nearest-neighbor search is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Next-token prediction

    Next-token prediction is a language-model concept about training behavior and measurement. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • No-repeat n-gram

    No-repeat n-gram is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Nucleus sampling

    Nucleus sampling is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • One-hot encoding

    One-hot encoding is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • One-shot prompting

    One-shot prompting is a language-model concept about representation and similarity. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Output constraint

    Output constraint is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Output context

    Output context is a language-model concept about instruction design and control. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Overfitting

    Overfitting is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Policy gradient

    Policy gradient is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Position embedding

    Position embedding is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Position interpolation

    Position interpolation is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Preference model

    Preference model is a language-model concept about evaluation design and failure analysis. It names a mechanism, representation, training practice, or operational behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Prefix-constrained decoding

    Prefix-constrained decoding is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Prompt compilation

    Prompt compilation is a language-model concept about generation behavior and sampling. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals

  • Prompt compression

    Prompt compression is a language-model concept about context selection and limits. It names a mechanism, representation, prompting pattern, decoding control, or context behavior that can change how an AI system processes input and produces output.

    LLM fundamentals