Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights

2609.02652 ↗ · 2026-09-02 · cs.LG

Pier-Jean Malandrino

21 pages, 5 figures. Preprint, not peer reviewed. Also deposited at Zenodo, doi:10.5281/zenodo.22133606

The finding

Headline claim

A full multi-shell Leech-lattice 2-bit weight decoder can be served on GPUs, but its unfolded in-VRAM format reads about 2.4x more bytes than QTIP's trellis code and runs 2.27x slower at batch-1 decode, so the strongest-quality 2-bit quantization carries a measurable serving-speed cost.

The test

Single NVIDIA L40S, decode-phase GEMV at batch 1 over the 252 projection matrices of Qwen3-4B, one process, ten arms (four bespoke layouts, AWQ 4-bit, QTIP 2-bit, FP16 and cuBLAS controls, a no-weights control), with every output row verified against an f64 reference before timing; end-to-end serving measured on 4B, 8B and 14B models, quality measured as perplexity and MMLU.

Effect size & scope

Planes14 layout reads 4.80 bits/weight at 2.15x FP16 speed; QTIP reads 2.00 bits/weight at 4.89x FP16, running 2.27x faster while reading 2.40x fewer bytes (0.91 vs 2.18 GB/token). End-to-end the path gains 1.11x, 1.29x, 1.41x at 4B/8B/14B; served 4B reaches 87.0 tok/s in 2.60 GB. Quality cost at 4B is 1.384x perplexity and 14.7 MMLU points, shrinking with size. Breaks down on a second card (A100) where every lattice arm falls below FP16, and below ~3.6 bits where decoding stops being shifts and masks.

Limitations

Authors concede launch geometry is not equalized between competing kernels, three model sizes support no scaling law, only one GPU (L40S) gives the headline numbers, the no-weights control shows their own launch geometry sets the timing floor (39% of the gap attributed to launch geometry), and QTIP bills no tail columns which understates their arms. Unstated: single model family (Qwen3), batch 1 only, no comparison against the original LLVQ authors' kernel directly at equal settings, preprint not peer reviewed.

Counterpoint

This is a one-card, one-model, batch-1 microbenchmark dressed in careful protocol; the deployed competitor (QTIP) dominates on every axis that matters for serving (bytes, speed, quality at similar rate), so the practical takeaway could be 'don't use Leech-lattice VQ for serving'. The 2.15x and 2.27x figures depend entirely on the authors' own port of QTIP and their own FP16 control. Reproducibility is strong (CSVs, pre-registered criteria) but the scope is narrow and no independent replication exists.

Economy implication

Little or nothing yet for the broader AI economy: this is an inference-efficiency engineering result showing that the best-quality extreme-quantization schemes may trade speed for memory footprint, relevant mainly to sovereignty-constrained deployments where a 70B model must fit on owned hardware (about 20 GB at 2 bits on disk). It marginally supports the claim that inference cost per unit capability keeps falling via format and kernel work, with savings captured by whoever controls the serving stack.

Who this matters for

Technical Capability: Shows the frontier of 2-bit quantization is now limited by serving-system costs (in-VRAM rate, kernel design), not by quantization math alone.

Market & Valuation: If extreme low-bit serving matures, it changes the hardware footprint needed to run capable open-weight models locally, affecting who can profitably serve models without renting hyperscaler GPUs.

Bears on (advisory · papers never write evidence)

[3.1] Inference cost per unit of capability is falling rapidly.

[6.2] Algorithmic / training-efficiency gains reduce the compute (and chip-access) needed for frontier capability.

Abstract

Leech-lattice vector quantization holds the strongest reported 2-bit quality under its own evaluation protocol. Its kernel decodes one shell; we found no implementation of the multi-shell decoder the rate requires. This paper supplies one and measures its serving cost for decode-phase GEMV at batch 1. First, a serving path for the full 301-class codebook: an offline expansion into GPU layouts and a fused dequantize-plus-matvec kernel reading them without warp divergence, verified against f64. Second, the in-VRAM rate is a design axis distinct from the on-disk rate. Four bit-exact layouts timed in one process show binary bit planes beating one-hot masks on size and speed at constant bandwidth (4.80 bits per weight, 2.15x FP16). Below 4.3 bits a second, irregular stream enters; at 3.6 the decode stops being shifts and masks. Third, deployed four-bit (AWQ) and two-bit (QTIP) GEMV kernels run in the same process. The trellis kernel reads 2.40x fewer bytes than our served layout and runs 2.27x faster at near-equal fractions of their byte bounds: the time gap tracks the traffic gap, the price of unfolding a codebook too large for a lookup table. Fourth, the validity envelope: the trellis kernel outruns our no-weights control, so our launch geometry sets that floor, and on a second memory hierarchy every lattice arm falls below FP16. With the output head held identical across arms, the kernel-and-format path gains 1.11x, 1.29x and 1.41x end to end at 4B, 8B and 14B; with an int8 output head the served 4B reaches 87.0 tok/s in 2.60 GB. The quality cost, 1.38x perplexity and 14.7 MMLU points at 4B, shrinks across the three sizes measured.