Mixed Precision Scaling Laws for Quantized Pretraining

2610.17637 ↗ · 2026-10-13 · cs.LG

Jinuk Kim, Rajarshi Saha, Muhammed Fatih Balin, Youngsuk Park

Preprint

The finding

Headline claim

A scaling law for quantized pretraining that adds persistent forward/backward optimization-error terms predicts held-out loss under mixed-precision FP4/FP8 configurations substantially more accurately than prior laws, and implies optimal precision should rise with compute budget.

The test

Ablation of FP4 recipe components (forward vs backward) on a 430M-parameter Qwen3 model trained on 12.8B tokens; a scaling-law fit grid spanning BF16, MXFP8, and several NVFP4 schemes with models up to 0.9B parameters and up to 35.8B tokens; validation on published Quartet Llama-style results up to 800 tokens per parameter; plus layer-wise mixed-precision experiments and compute-optimal allocation solves.

Effect size & scope

Up to 11x lower MSE in held-out loss prediction versus existing scaling laws. Component ablation: forward quantization costs +0.94-1.22% loss vs backward-only +0.32-0.72%; the best full-FP4 recipe (Quartet II) still trails BF16 by roughly +0.36% at 430M/12.8B scale. All results are at sub-billion scale; extrapolation to frontier sizes is asserted by the law, not demonstrated.

Limitations

Authors concede even the best FP4 configuration leaves a noticeable gap to BF16, motivating mixed precision rather than uniform 4-bit. Unstated: all Qwen3 experiments are tiny relative to frontier runs; the persistent-error term is validated only in this regime; compute-optimal conclusions depend on assumed hardware speedups (NVFP4 vs BF16 speedup is a free parameter in the figures); convex-theory sketch does not strictly apply to LLMs. This is a preprint with internship-affiliated authors.

Counterpoint

The 11x MSE improvement is on small-model held-out splits where scaling-law fitting is easy to overfit; grid coverage is narrow (one model family plus one published external grid); the headline 'optimal precision rises with compute' conclusion is an extrapolation driven by assumed speedup constants rather than measured frontier-scale training; no release of code confirmation is stated.

Economy implication

If it holds at scale, low-precision mixed formats could cut pretraining compute cost per unit capability, feeding the falling-cost side of training economics; but the paper itself shows 4-bit-everywhere is not yet loss-free, so near-term savings are incremental, not a step change.

Who this matters for

Technical Capability: Provides a sharper quantitative tool for deciding forward/backward and per-layer precision, directly relevant to how much cheaper frontier-scale pretraining can get.

Market & Valuation: Materially better quantized-training economics would shift the compute-cost lines in lab unit-economics and hyperscaler capex-efficiency arguments, though only as a forecast input at this stage.

Bears on (advisory · papers never write evidence)

[6.2] Algorithmic / training-efficiency gains reduce the compute (and chip-access) needed for frontier capability.

[3.1] Inference cost per unit of capability is falling rapidly.

Abstract

Since scaling laws were introduced for modern LLMs (Kaplan et al., 2020), several works have extended them to account for quantization, tracking the emerging low-precision data types supported on AI accelerators. We develop a more accurate scaling law for quantized pretraining with mixed precision across the forward and backward passes, grounded in optimization theory and validated empirically. Our formulation separates the effects of quantization on model capacity and on optimization efficiency, and explicitly models the persistent optimization error from forward and backward quantization identified through convergence analysis. To select the regimes for fitting and evaluating our law, we first decompose existing 4-bit training recipes into their forward and backward components, isolating those that most affect loss. Our Qwen3 experiments under selected 4-bit regimes, together with published Llama-style experiments, show that our law predicts loss with up to $11\times$ lower MSE than existing laws as model size and token count scale. Finally, we extend our law to layer-wise mixed precision and identify the optimal quantized-pretraining recipe, i.e., model size, training tokens, and per-layer forward/backward precision, under a hardware-aware compute budget. Unlike prior precision scaling laws, our law predicts that the optimal precision increases with the compute budget.