# KV Cache

> The KV cache stores each token's attention keys and values so an LLM doesn't recompute the whole context per new token — the memory that makes generation fast.

The KV cache stores the attention keys and values for every token already processed, so each new token attends over cached history instead of reprocessing the whole context. It trades quadratic waste for memory that grows with context length and batch size, which is why long-context serving exhausts VRAM before compute.

**The KV cache is the stored attention state — the key and value vectors for every processed token — that lets a transformer generate each new token by attending over cached history instead of recomputing the entire context from scratch.**

Without it, producing token N would mean reprocessing all N−1 prior tokens — quadratic waste. With it, [inference](/glossary/inference) splits cleanly: prefill computes the prompt's KV once, then each decode step computes one new token's attention against the cache. The trade is memory: KV state grows with context length × batch size, which is why long-[context](/glossary/context-window) serving exhausts VRAM before compute, why serving engines like [vLLM](/tools/vllm) build their architecture around KV memory management, and why KV quantization and eviction schemes are an active frontier.

Two product-level features sit on top of it: [prompt caching](/glossary/prompt-caching) persists prefix KV state across requests for cost savings, and [speculative decoding](/glossary/speculative-decoding) exploits cheap cache-backed verification to accept multiple drafted tokens per step.

---

_Source: https://agentscamp.com/glossary/kv-cache — Term on AgentsCamp._
