A reference from Weave

Find a term

Understand how AI systems are tested, scored, and compared. Learn to distinguish a useful evaluation result from a number that does not transfer to your workload.

Terms beginning with P

13 terms

  • Pairwise ranking

    Pairwise ranking is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Pairwise win rate

    Pairwise win rate is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Pass at k

    Pass at k is the probability that at least one of k generated samples solves an evaluation task. It measures the benefit of giving a model several attempts, rather than the reliability of its first answer.

    Evaluations and benchmarks

  • Pass at one

    Pass at one is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Pass-fail evaluation

    Pass-fail evaluation is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Perplexity

    Perplexity is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Point-based grading

    Point-based grading is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Position bias

    Position bias is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Precision

    Precision is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Preference rate

    Preference rate is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Privacy evaluation

    Privacy evaluation is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Production evaluation

    Production evaluation is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks

  • Prompt sensitivity

    Prompt sensitivity is a defined lens for examining AI system behavior with a stated task, evidence, and interpretation rule.

    Evaluations and benchmarks