Positional task conditioning for scalable defect detection across product families in large product catalogs

A small finetuned model with Positional Task Conditioning (inserting task tokens at three structural prompt boundaries) can match a frontier LLM on catalog defect detection at up to 98% lower inference cost, and divide-and-conquer task decomposition lifts frontier F1 from 52% to 87% on this task.

Amazon Catalog AI system: detect four defect types (overstuffing, unit systems, duplicates, theme) in product families, as four three-class sub-tasks on variating attributes. Teacher: Claude Sonnet 4.5 monolithic vs. divide-and-conquer prompting. Students: five open models (Qwen3 8B/14B/32B, Mistral-Nemo 12B, Mistral-Small 24B) finetuned via QLoRA on 73,723 family-attribute pairs with teacher labels (98.13% unanimity retained). Human-labeled eval set of 2,296 pairs (390 defective) plus 11,130 synthetic pairs; 7-configuration token-placement ablation.

Monolithic Claude: F1 0.521; divide-and-conquer Claude: F1 0.876 (precision 0.972, recall 0.797). PTC reaches within 1.79% F1 of the frontier teacher and beats rationale-based distillation across all five students and two architecture families; multi-position token placement beats single-prefix conditioning. Results shown on one company's catalog domain; cost saving cited as up to 98% vs frontier inference.

Authors note no public datasets exist for these tasks (all evaluation is proprietary or synthetic), certain error types are rare in real data (unit-systems and theme have only 50-57 positive examples in the human test set), and results are single-domain (e-commerce catalogs, English-language countries). Unstated: the teacher labels the training data, so student 'near-parity' partly measures distillation fidelity to the teacher, not ground truth; the human-labeled set is small and skewed toward overstuffing; deployed-system claims (10M+ families) are not independently validated.

This is corporate engineering work with a method name attached: the headline comparison (PTC vs RBD) is a narrow ablation in one vertical, the human test set has tiny positive counts for two of four tasks, and synthetic evals with injected errors are easier than natural defects. The 98% cost saving depends on unstated frontier API pricing. No external replication is possible without the proprietary data.

In the next 12-24 months this is mainly evidence that domain-specific small-model distillation from a frontier teacher is a repeatable pattern for high-volume enterprise operations, shifting unit costs of structured classification work by roughly an order of magnitude. It says little or nothing yet about frontier model economics or labor displacement directly; it does suggest enterprises with large proprietary data pipelines can substitute finetuned open-weight students for frontier API calls on narrow tasks.

Technical Capability: Practitioners should note that explicit positional task conditioning beats rationale-only distillation for multi-task small models, a cheap technique transferable beyond catalog QA.

Labor & Knowledge Work: Back-office catalog quality and data-cleaning work is a concrete, automatable task class where a 70-person-equivalent accuracy bar is met by a small model at a fraction of cost.

Market & Valuation: The pattern of frontier-teacher-to-small-student distillation in production supports the view that narrow enterprise workloads migrate off frontier APIs, pressuring frontier vendors' inference revenue.