A training technique that teaches a smaller student model to mimic the outputs of a larger teacher model, producing compact models with disproportionately high capability.
Broad technical consensus on what this term means. Disagreements that invoke it are about the world, not the word.
Distillation allows a smaller, cheaper model to approach the performance of a much larger one by learning from the teacher's output distribution rather than raw data alone. It is a core mechanism behind training-efficiency gains and is why open-weight models can reach near-frontier capability at a fraction of the original training cost. The scaffold references training efficiency and open-weight parity but has no concept explaining the technique that most directly connects them.