DiffusionGemma Technical Report

2608.00146 ↗ · 2026-07-31 · cs.CL, cs.AI · h-max 31 · 0 citations

DiffusionGemma Team, Adrien Ali Taïga, James Assiene, Daniele Calandriello, Rahma Chaabouni, João Gante, Tamara von Glehn, Nate Keating, Chris Knutsen, Martin Kukla, Tianlin Liu, Ivan Lobov +32

The finding

Headline claim

A discrete-diffusion fine-tune of Gemma 4 26B (DiffusionGemma) generates roughly 1,500 tokens per second on a single H100, achieving around 20 tokens per forward pass versus 3-6 for state-of-the-art speculative decoding, while retaining competitive benchmark quality on GPQA-Diamond and LiveCodeBench.

The test

The model was evaluated on GPQA-Diamond and LiveCodeBench-v6 for quality, and throughput was measured on a single NVIDIA H100 GPU (FP8, batch size 1) and on NVIDIA RTX 6000 (third-party). Speed comparisons were made against Gemma 4 AR family models (E2B to 31B, with and without multi-token prediction), Nemotron 14B, LLaDA 2.1 Flash 100B, and Mercury 2. Training used fewer than 10% of the AR model's original training token budget across two stages: SFT on bidirectional denoising, then a combined sampler distillation plus RL phase.

Effect size & scope

Approximately 1,500 TPS on a single H100 at batch size 1, versus 3-6 tokens per forward pass for speculative decoding; roughly 20 tokens per forward pass averaged, typically achieved in about 12 denoising steps out of a maximum of 48. Third-party testing showed up to 2,000 TPS on RTX 6000. Quality results (exact scores) are reported as averages across GPQA-Diamond and LiveCodeBench-v6 but the paper presents these in a Pareto plot rather than as explicit numeric accuracy figures in the excerpt provided. The model claims a new Pareto frontier above prior open-weight diffusion models and the full Gemma 4 AR family on the speed-quality tradeoff.

Limitations

Authors acknowledge that adapting to the diffusion regime introduces a performance penalty versus the original AR baseline. The factorized denoising process assumes conditional independence per reverse step, which can introduce local inconsistencies or conflicting grammatical predictions. Evaluation is limited to two benchmarks for the headline Pareto claim. The model is described as experimental. Third-party throughput figures come from a single external provider. Full training compute details and ablations are not fully disclosed in the excerpt.

Counterpoint

This is a Google DeepMind technical report, not a peer-reviewed study, so claims about a new Pareto frontier are self-assessed. The quality comparison relies on only two benchmarks, which may not capture general capability degradation from AR to diffusion fine-tuning. Throughput measured at batch size 1 is a best-case scenario for diffusion's memory-bandwidth advantage; at high concurrency, AR batching narrows the gap. The base model (Gemma 4 26B MoE) is already a strong starting point, making it unclear how much of the quality result is architecture versus diffusion training. No third-party independent reproduction of quality or speed claims is provided.

Economy implication

If throughput gains of 5-10x over speculative decoding hold at production batch sizes, inference cost per token for latency-sensitive, low-concurrency serving (edge devices, single-user apps) could fall materially within 12-24 months as the open-weight model is adopted and optimized. However, at high-concurrency enterprise serving where AR batching is efficient, the economic advantage narrows. The open Apache 2.0 license accelerates commoditization of fast inference, pressuring proprietary diffusion API providers like Mercury.

Who this matters for

Technical Capability: Demonstrates that fine-tuning an existing AR MoE model into a discrete diffusion model with under 10% of original training compute can shift the speed-quality Pareto frontier meaningfully, opening a new design axis for fast text generation.

Market & Valuation: An open-weight model achieving 1,500+ TPS on a single H100 under Apache 2.0 directly pressures proprietary fast-inference API providers and reduces the moat of speed-differentiated products like Mercury.

Regulatory & Legal: Open-weight release under Apache 2.0 of a high-throughput model capable of multimodal inputs and thinking mode raises questions about governance frameworks for rapidly deployable, high-capacity open models.

Bears on (advisory · papers never write evidence)

[1.1] Frontier benchmark scores kept rising gen-over-gen through 2025–26.

[1.3] Agentic reliability (multi-step completion without human correction) is improving materially, not just peak single-shot.

[3.1] Inference cost per unit of capability is falling rapidly.

[5.2] Open-weight models reach "good enough" parity for most economic use cases.

[B3] Training-efficiency gains will reduce aggregate compute demand (threatening "more compute = more value").

[B4] Model commoditization determines where rent settles (away from labs).

Abstract

We introduce DiffusionGemma, an experimental open-weight language model that uses discrete diffusion to generate text at exceptionally high speed. Rather than decoding one token at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel, avoiding the sequential decoding bottleneck of conventional autoregressive (AR) large language models. Instead of training from scratch, we obtain DiffusionGemma by fine-tuning the mixture-of-experts Gemma 4 model with 3.8B activated and 25.2B total parameters. Our compute-efficient two-stage training pipeline uses fewer than 10% of the starting AR model's total training token budget. The first stage uses supervised fine-tuning to teach bidirectional denoising, while the second stage combines reinforcement learning with sampler distillation to jointly improve generation quality and inference efficiency. DiffusionGemma establishes a new Pareto frontier for the trade-off between generation speed and model capability. Averaged across our full evaluation suite, it generates around 20 tokens per forward pass and achieves roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, which is substantially faster than AR models even with state-of-the-art speculative decoding. DiffusionGemma also retains the starting model's support for thinking mode, multimodal inputs, and long contexts. Despite diffusion fine-tuning, it remains capable of AR generation with only minor performance degradation, suggesting a path toward hybrid diffusion-AR decoding.