ContextPipe: Database-Inspired Context Assembly for Long-Horizon Agents

2609.00749 ↗ · 2026-09-01 · cs.AI, cs.DB

Peng Xu, Zuyu Zhang, Yuze Sun, Feng Tian, Long Wang, Chen Zhang

The finding

Headline claim

A database-style, five-phase context assembly runtime (ContextPipe) reduces token volume, LLM calls, and latency versus append-only context construction in long-horizon agents, while making context decisions auditable and replayable.

The test

Preliminary evaluation on the SWE-bench Pro Qutebrowser subset only, comparing ContextPipe against an append-only baseline on token volume, LLM call count, response time, and KV cache-hit ratio; system paper from MatrixOrigin and Tsinghua presented at a VLDB workshop.

Effect size & scope

Total tokens down 31%, LLM calls down 23%, response time down 9%, at the cost of a lower KV cache-hit ratio. Reported on one narrow benchmark subset, so no evidence the gains generalize across tasks, models, or providers.

Limitations

The authors themselves label the evaluation preliminary; only one benchmark subset and one baseline are used, and they concede the response and tool-schema reserve terms (Rt, Rs) are left at zero pending future work. Obvious unstated gaps: no task success-rate or quality metric is reported, so token savings could come with degraded completion; no cost comparisons across providers; no code or end-to-end task-level win-rate evidence in the provided text.

Counterpoint

This is a systems/position paper with an n-of-one-subset evaluation: a 31% token cut on a single subset could reflect the baseline being a naive append-only strawman, and lower cache-hit ratio may erode real cost savings since cached-prefix tokens are priced far below uncached ones. Without quality or success-rate numbers, 'cheaper' is not 'as good.'

Economy implication

If validated, context assembly becomes an engineering discipline with measurable cost levers, meaning agent inference cost per task is partly reducible at the orchestration layer rather than by waiting on cheaper tokens. But this single preliminary result supports at most a small inference-cost-curves data point; it moves little in the economics debate for the next 12 to 24 months on its own.

Who this matters for

Technical Capability: Agent builders get a concrete, auditable architecture for context assembly that quantifies the token and latency savings available beyond naive prompt stacking.

Market & Valuation: Signals an emerging infrastructure-layer (agent runtime middleware) where vendors could capture margin independent of model providers, worth watching for consolidation.

Abstract

Long-horizon large language model (LLM) agents require context assembly: the runtime must decide what to include in each prompt, in what order, and when to compact history under a hard context-window budget and a byte-sensitive prompt cache. In production agentic systems, this logic is scattered across prompt builders, ad hoc compaction routines, cache-break workarounds, and per-provider shims. We argue that context assembly is structurally isomorphic to query execution in a relational database: both execute under a hard budget, exploit a tiered cache, and leverage statistics. We adopt this discipline in ContextPipe: a five-phase pipeline (Plan Bind Optimize Execute Feedback) backed by a structured data-source catalog, a deterministic cache-aware optimizer, and an EXPLAIN ANALYZE trace. We show that context in ContextPipe is auditable, replayable, and failure-isolated. A preliminary evaluation using the SWE-bench Pro Qutebrowser subset shows that, compared with the append-only context construction policy, ContextPipe reduces total token volume by 31%, LLM calls by 23%, and response time by 9%, at the cost of a lower KV cache-hit ratio.