The compute-heavy first phase where a model learns general capabilities by predicting tokens across vast text corpora.
Broad technical consensus on what this term means. Disagreements that invoke it are about the world, not the word.
Pre-training is the giant first pass: the model reads an enormous corpus and is trained on one simple objective, predict the next token. Done at sufficient scale, that objective forces the network to absorb grammar, facts, idioms, code, and reasoning patterns, because they all help prediction.
It is where almost all the compute (and money) goes, and it produces a raw "base model": broadly capable but unsteered. Pre-training is the phase scaling laws describe, and the phase people mean when they say a model "cost hundreds of millions to train."