Explainers: AI inference

KV cache explained: how it speeds up AI inference

A KV cache reuses attention keys and values during generation, saving compute while using more accelerator memory.