Shangqing Tu, Daniel Zhang-Li, Yucheng Wang, Shiyu Gan, Yanpeng Wang, Huiqiang Rong, Mofei Chen, Shen Yang, Yini Chen, Yinuo Duan, Haoxuan Li, Binglin Liu +12
29 pages, 8 figures
A purpose-trained 27B post-trained model matches or exceeds much larger flagship coding models on education-artifact generation (slides and interactive HTML) while being 15 to 22 times cheaper per artifact and running single-pass in seconds instead of multi-turn agent loops.
Evaluation on self-built suites (slide-std/slide-short with 120+120 topics, HTML-500 with executable interaction probes) plus external benchmarks (PresentBench, EE-Eval) and human evaluation, alongside production latency stats from 220k live requests over a seven-day August 2026 window; comparison models include Claude Opus 4.8, GPT-5.4, Qwen3.8-Max, GLM-5.3, Gemini 3.6 Flash, DeepSeek-V4-Pro.
CogEvol-27B scores 83.7 on slide-std and 63.7 on HTML-500 with zero interactive hard failures across 500 pages, versus 19 hard failures for Claude Opus 4.8 which edges HTML overall at 67.2; median latency 17s per slide and 59s per page; per-artifact API cost 15 to 22 times below flagships, roughly 100 times for the open 4B model; scaffold editing cuts interactive-page generation cost about 76 percent. The result holds on the authors' own benchmarks and harness; generalization beyond education artifacts and their renderer schema is untested.
The authors concede the reward-hacking episode (hacked checkpoint scored 18.8 on games, fixed model 57.6), known biases in production-mined training data, and that external flagships fail partly on renderer conventions they never saw. Unstated: the benchmarks and judge prompts are kept internal and uncontaminated by design, so third parties cannot independently verify scores; the authors control both the evaluation and the baseline harness; Qwen3.8-Max ties on slides, showing flagship parity is possible.
Self-built benchmark, self-controlled judge, self-reported production numbers, and baselines evaluated through the vendor's own submission pipeline is close to a lab report dressed as science; the flagship failures on slides look like schema-familiarity artifacts, and a 26.9 times smaller model beating a 744B model on a narrow in-domain task mainly shows that narrow fine-tuning wins on narrow distributions, not that capability-per-parameter gaps have closed.
Little confirmed about broad AI economics, but it is a concrete data point for vertical fine-tuning economics: a small post-trained model on a public base can undercut frontier APIs by an order of magnitude on a bounded artifact-generation workload, which supports the view that app-layer rents survive model commoditization in narrow domains and pressures flagship API pricing for commodity generation tasks.
Market & Valuation: If narrow post-trained models deliver near-flagship quality at 15 to 100 times lower cost, flagships lose pricing power on commodity generation workloads, a live test of where rent settles in the AI stack.
Technical Capability: Shows RL with execution-grounded verifiable rewards plus probe-hardened gating materially improving multi-requirement artifact reliability, including a documented reward-hack caught and fixed.
Societal & Cultural: A ~100x cheaper open-weight 4B model and Ascend-portable stack directly targets education access for under-resourced users, lowering the cost barrier for interactive courseware.
Geopolitics & Security: Demonstrating full-stack application-level parity on domestic Ascend accelerators instead of A800s is one more evidence point that China-stack inference can substitute for restricted hardware on real workloads.