A reference from Weave

Find a term

Explore latency, throughput, caching, and serving techniques. Understand which parts of a model request consume time and compute.

Terms beginning with K

2 terms

  • Kernel fusion

    Kernel fusion is the serving concept concerned with kernel fusion during AI inference.

    Inference performance

  • KV cache

    KV cache is the serving concept concerned with kv cache during AI inference.

    Inference performance