TACIT-Switch: Cost-Aware Model Escalation for LLM Agents from Censored Supervision

2608.27911 ↗ · 2026-08-28 · cs.LG

Ji'an Lei, Jian Huang

17 pages, 6 figures, 3 tables, 1 algorithm

The finding

Headline claim

A survival-analysis-style routing policy that permanently hands off from a cheap to a strong model, trained from censored teacher annotations, achieves higher held-out agent success at comparable inference cost than task-level, step-level, and fixed-prefix routing baselines.

The test

Controlled statistical simulations (parameter recovery at n=500/2,000/8,000 and a mechanism-based multi-step simulation with ablations), plus interactive-agent experiments on ALFWorld (4B and 9B cheap backbones) and DABench, comparing against task-, step-, and fixed-prefix routing under a shared cost-accounting protocol.

Effect size & scope

In the mechanism-based simulation, TACIT-Switch improves success by 7.4 to 11.1 percentage points over the three routing baselines at comparable cost. On ALFWorld it reaches 48.5% held-out success with a 4B cheap model and 45.5% with a 9B cheap model; on DABench 73.1%, in each case the highest among learned policies with operating points selected on development data. These numbers are benchmark-scale; the full text provided cuts off before the complete ALFWorld/DABench comparison tables.

Limitations

The authors concede the teacher annotation mechanism may not be ignorable (they use a working likelihood, not a generative claim), the cumulative-risk scale is a decision construct rather than a behavioral model, and thresholds must be tuned per cost budget on development data. Unstated: ALFWorld and DABench are synthetic-ish benchmarks far from enterprise workflows; only a two-model (Cheap/Strong) cascade is tested; the method needs paired full rollouts plus teacher annotation for training, so the labeling pipeline cost is not priced into the claimed savings.

Counterpoint

Only two benchmarks and a simulated setting; baselines are three routing policies the authors chose and implemented under their own protocol, excluding methods like TRIM or EvoRoute which they dismiss as incompatible rather than benchmarked; results depend on hand-engineered diagnostic features (repeated actions, stalls) that may not transfer; the 48.5% ALFWorld success rate is low in absolute terms, and the headline gains partly reflect a carefully selected operating point alpha.

Economy implication

If it generalizes, cost-aware escalation like this strengthens the case that most enterprise agent workloads will run on small cheap models with occasional expensive rescue, pushing inference spend per task down and weakening the claim that frontier-model access is needed per-token for agent products. For the next 12 to 24 months this is a method paper, not direct economic evidence; it supports inference-cost-reduction as a mechanism but demonstrates nothing yet about realized enterprise ROI or revenue.

Who this matters for

Technical Capability: Routing and escalation researchers get a new control point (permanent handoff timed by censored survival evidence) with a clean statistical formulation and a finite-sample stability result.

Labor & Knowledge Work: If cheap-model-plus-escalation becomes the default agent stack, the productivity economics of automating knowledge work rest on small-model reliability plus selective escalation, changing which model tiers actually get consumed at scale.

Bears on (advisory · papers never write evidence)

[3.1] Inference cost per unit of capability is falling rapidly.

Abstract

Agents with smaller language-model backbones are less expensive but can drift into persistent failure modes, whereas those with larger backbones are generally more reliable but more costly. This reliability-cost trade-off motivates routing methods that decide when to invoke an agent with a larger backbone: before execution, after a fixed trajectory prefix, or locally at individual steps. Our method, TACIT-SWITCH, learns permanent handoff policies from accumulated trajectory evidence and Teacher-Annotated Censored Intervention Times (TACIT). It represents each annotation as an interval-censored observation on a cumulative-risk scale. The resulting mixture-cure threshold model estimates the probability that the paired Strong rollout succeeds and, conditional on success, the handoff threshold; no teacher is required at deployment. In a mechanism-based multi-step simulation, TACIT-SWITCH improves success by 7.4-11.1 percentage points over task-level, step-level, and fixed-prefix routing baselines at comparable cost. Within that controlled simulation, ablations show that task features and cumulative trajectory risk provide complementary information. With operating points selected on development data, TACIT-SWITCH achieves the highest held-out success among learned policies on both ALFWorld (48.5% with 4B Cheap; 45.5% with 9B Cheap) and DABench (73.1%).