Stored keys and values for tokens already processed, so generating each new token does not recompute the whole sequence.
Broad technical consensus on what this term means. Disagreements that invoke it are about the world, not the word.
Generation is one token at a time, and each new token attends to everything before it. Recomputing keys and values for the entire history on every step would make the tenth token roughly ten times as expensive as the first. Instead they are computed once and cached.
This is why serving cost scales with context length in memory rather than in repeated compute, and why long-context serving is a memory-capacity problem as much as a compute one.