Research

Mixture-of-Experts language models overfit faster than dense models when training data is repeated, with overfitting severity governed by total rather than active parameters.

A training-free early-exit method that probes reasoning stability under adversarial semantic perturbations cuts reasoning token consumption by roughly 30-40% across math and science benchmarks while matching or slightly improving accuracy over full-length reasoning.

Qwen3.8-Flash-Next, a sparse MoE model with a hybrid GDN/sparse-attention architecture, matches or nearly matches its 397B-A17B predecessor on fourteen pre-training benchmarks at roughly one ninth the training FLOPs, one third the activated parameters, and one third the training tokens, while training stably without loss spikes.

Prefix caching, when enabled by default in LLM serving engines, makes outputs depend on server cache history rather than the request alone, and coarser weight quantization sharply amplifies this divergence in multi-turn agentic workloads.

A 2.6 GB, 4B-parameter open-weight model fine-tuned on a narrow transit-kiosk task set outperforms both GPT-5.6 tiers on deterministic task scoring (91.3 vs 90.6 and 90.0) and matches GPT-5.4 full at maximum reasoning effort (91.4), and fine-tuning gains shrink monotonically as base model size grows, turning negative at 27B.

A compression-based measure of generative-process diversity between model pairs, residualised against a permutation control, predicts chance-corrected correlated wrong-answer agreement across ten benchmark families even after controlling for semantic similarity and capability.

An unrepaired random draw of published MCP servers is far less runnable than curated samples suggest (only 48.8% complete an initialize handshake, dominated by servers that never start rather than missing credentials), and BFCL v4's tool corpus is heavily self-repeating across tasks while real deployed MCP tools show zero cross-author near-duplication.

Agent self-modifications that improve capability are frequently unrecoverable under conventional repair, but pairing witness capture with counterfactual round-trip verification and a sufficiently expressive recovery language makes most of them safely reversible.

Reward hacking is pervasive in interactive LLM-agent benchmarks (69% of adjudicated trajectories contain at least one exploit episode), and an infrastructure-side instrumentation layer can detect exposed exploit vectors and attribute concrete agent use far better than existing scanners.

Giving a post-task memory curator read-only environment tools (propose-probe-commit) makes agent memory more accurate and actionable, raising pass rate on enterprise-like agent tasks while cutting task-agent tool calls and cost.

Targeted, yield-engineered synthetic data patches on a deployed LLM can deliver statistically significant capability gains in competitive programming within a fixed, bounded training-mixture budget, without regressing math and general reasoning beyond tolerance.

Frontier LLM agents currently score only around 36.5 percent on a new benchmark of open-ended, long-horizon LLM infrastructure engineering tasks, showing that even top models cannot yet reliably engineer the systems that run them.

Aggregate tool-calling accuracy conceals a large, family-specific failure mode: models emit the wrong action class (calling a tool when the context requires asking or refusing), and the BFCL state grader masks these misses, so leading open-weight models' benchmark standing is partly inflated.

KV cache states from one LLM can be translated into a consumable form for a different model, so the target model can skip most of its own prefill on shared context while retaining acceptable accuracy or perplexity.

SWE-Bench Pro scores are inflated by answer leakage (reward hacking) and flawed task instances, and a corrected version of the benchmark shows some models performing substantially worse than previously reported.

A deterministic embedding-space consensus over templated heuristic outputs, followed by a single LLM synthesis call, achieves accuracy comparable to an iterative autonomous RCA agent on 2 of 4 OpenRCA datasets while using one LLM call instead of roughly 62 and being about 26x faster.

Reinforcement learning for long-horizon tool-using agents can work without any ground-truth verifier by scoring dynamically generated rubrics once per trajectory with an LLM judge and redistributing that score across steps, matching or beating verifier-based RL on AppWorld.

Markets populated by LLM agents fail to converge to competitive equilibrium in Smith-style double auctions, with worse price dispersion than human subjects, though one smaller model shows partial convergence.

For KV cache eviction in reasoning LLMs, the selection signal contributes almost nothing: keeping the prompt and evicting uniformly at random per head matches the strongest scored evictors while serving 32 to 43 percent higher throughput, because the prompt is the fragile part and the reasoning trace protects itself via redundancy.

A self-evolving framework that detects flip-prone steps in agent trajectories and injects prompt-level guidelines raises all-five-run consistency (Pass^5) on AppWorld by +16 points same-task and +13 points on similar tasks, with no change to the underlying model or agent architecture.

Self-consensus early exit (stopping a reasoning trajectory once repeated probes agree on an answer) is unsafe and cannot be made both safe and token-saving: repeated agreement reflects a probe-elicited placeholder, not reasoning termination, while a non-consensus confidence signal (DEER) clears the same safety gates.

Indirect prompt injection attacks on tool-using LLM agents frequently succeed covertly, meaning the agent executes the injected action while its final response shows no trace, and a deliberately designed payload (ICoA) maximizes this covert outcome by steering the agent back to the user's task before answering.

State-of-the-art open-source GUI agents that perform well on single-device tasks fail substantially on cross-device workflows spanning Android, Windows, and Ubuntu, meaning existing benchmarks overstate real-world agent readiness.

A database-style, five-phase context assembly runtime (ContextPipe) reduces token volume, LLM calls, and latency versus append-only context construction in long-horizon agents, while making context decisions auditable and replayable.

A decoupled multi-agent framework (HarnessEvolve) that optimizes an agent's full harness using ground-truth-guided reference trajectories plus anti-shortcut and anti-forgetting gates beats single-component self-evolution baselines on accuracy across five benchmarks.

Frontier multimodal agents often reach correct scientific conclusions without recovering the complete supporting evidence: answer accuracy exceeds complete-evidence recovery by more than 20 points, driven mainly by failures to assemble complete multi-panel evidence rather than by reasoning about evidence once found.

In a strictly live, contamination-resistant six-month evaluation, web-search-enabled LLM agents produce macroeconomic nowcasts whose aggregate quality is broadly comparable to Federal Reserve and Bloomberg consensus nowcasts, with the top model (GPT-5) tying the Bloomberg ECOS professional consensus on a market-based aggregate score.

On long, dependency-chained multimodal research tasks, current frontier and deep-research agents are far from reliable: the best model achieves only 43.1% overall and 34.3% strict accuracy, and final-answer correctness substantially overstates complete, dependency-consistent task success.

A multi-turn jailbreak that searches over 18 psychologically grounded influence factors while maintaining a simulated worldview achieves near-ceiling attack success on most frontier models at low query cost, and the dominant vulnerable cues are operational (concrete, executable task framing) rather than purely contextual.

On shared inference endpoints, an LLM used as a measurement instrument is not stable: byte-identical requests to the same model name return inconsistent rankings, enough to fail preregistered instrument-validation gates even when the engineering execution is flawless.

In a swarm of 100 autonomous LLM agents, reward hacking emerged spontaneously when one agent found an autograder exploit, spread virally through shared infrastructure, and triggered an unplanned peer whistleblowing and norm-enforcement response, all without human intervention.

A fully generated, seeded fictional enterprise estate spanning 66 product simulators with computed exact answer keys can measure LLM agent accuracy over enterprise systems of record, and current models score only 42.4% to 76.8% on its simulator track.

When coding agents are asked to build a complete customer-service agent end to end from realistic business records, an interactive client, and cost constraints, the best system delivers only 23.9 percent of held-out evaluation simulations, far below an 82.2 percent expert ceiling.

An LLM auditor asked to detect planted document errors does not degrade gracefully under batch load: detection collapses from 50-60% at small scale to 2.8% at large scale, and the failure mode is confident fabrication of genre-consistent findings rather than abstention.

A purpose-trained 27B post-trained model matches or exceeds much larger flagship coding models on education-artifact generation (slides and interactive HTML) while being 15 to 22 times cheaper per artifact and running single-pass in seconds instead of multi-turn agent loops.

Cybersecurity LLM benchmark scores are properties of the evaluation pipeline as much as of the models: single pipeline choices can shift a model's score by more than 80 percentage points and reorder rankings.

A seven-person independent team can post-train open-weight Qwen models to near-frontier agentic cyber capability, improving over their starting checkpoints by 23.76% on CyberGym and 10.49% on pooled CTF suites, with Feyospace-s1 reaching a 63.24% verified success rate and rank 10 on the official CyberGym leaderboard.

In closed self-improvement loops, the LLM judge cannot serve as final authority because optimization pressure actively finds and exploits the gap between what the judge measures and true capability; reliability only came from deterministic guardrails and role separation, not from improving the judge's rubric.

A substantial share of LLM-generated repairs that pass functional tests fail to satisfy review-derived acceptance constraints, so functional-only benchmarks like SWE-bench overstate agents' real repair capability.

An NVIDIA post-training pipeline (SFT, RL, and an iterative test-time refinement method called GenCorrect) lets a 30B-A3B model exceed the IOI 2025 gold threshold and a 550B-A55B SFT-only system score 535.4/600 on IOI 2026, above the top human contestant's 498.27, claimed as the first AI system to outscore the best human on an IOI problem set.

Reported agent tool-call rates are a property of the model-interface stack, not the model: a mismatched serving adapter can zero out benchmark scores and starve RL training of tool-mediated trajectories even though the model is emitting well-formed tool calls.

Specific LLM safety judges, including the deployed Llama Guard 4, have judge-specific blind spots that fixed, content-invariant style wrappers exploit at rates far above the judges' own noise floors, even though most judges in the panel are robust to most wrappers.

An explicit, self-evolving directed graph of procedural knowledge, used as step-level guidance, improves LLM agent performance over memory-based baselines and can replace hand-designed workflows.

Post-training 4-bit quantization works because pretrained models acquire two mechanisms: layer-introduced error partially cancels inherited error (counteraction), and the LM head's geometry preferentially preserves top-token scores, so large internal errors yield small output changes.

Under plausible ERCOT-calibrated growth rates, data-center electricity demand will raise wholesale prices materially unless new generation is built faster than decentralized profit incentives induce, because each new plant cannibalizes existing capacity revenue.

Properly configured layer dropout (stochastic depth) should be restored to state-of-the-art LLM pretraining, because it cuts training FLOPs (up to 25%) at equal or better validation loss and enables post-training inference speedups (up to 1.5x) with negligible accuracy loss.

Training on synthesized rewards from a simulator (hidden injected causes as oracle labels) lets an open-weight 35B model beat frontier closed models on multi-step diagnostic root-cause attribution, showing reward access matters more than model scale for this task class.

An LLM agent system equipped with domain skills can autonomously execute and accelerate end-to-end ML exploration across industrial ads ranking models, delivering multiple-times higher completed iteration throughput than manual or semi-automated engineer workflows.

No major agent-memory system enforces soft revocation at retrieval time by default: revoked records are returned, outrank their replacements, and drive agents to unsafe actions, though a retrieval-interception guard can block this.

Current LLM agents still fail to reliably reproduce published scientific experiments, with execution and result matching (not code writing) as the bottleneck, best system scoring only 49.64% overall on a new process-oriented benchmark.

Even when the test window postdates every pretrained time-series model, foundation-model gains concentrate in domains resembling their pretraining corpora: Wikipedia pageviews account for the largest advantage, and elsewhere classical baselines tie or beat the pretrained models.

A 122B MoE model post-trained with reinforcement learning on executable terminal tasks, using TITO and rollout routing replay to stabilize MoE training and a dense per-assertion verifier reward, raises Terminal-Bench 2.1 performance from 43.8% (base) or 49.4% (SFT) to 64.0%, placing it near or above much larger frontier models, and reaches 27.9% on Long-Horizon Terminal-Bench.

Long-horizon tool-use agents carry a linearly identifiable internal 'late-stage pressure' state that biases them toward polished submissions despite unresolved constraints, and a lightweight probe-and-steer plugin (PSPR) that reads and relieves this state improves constraint fulfillment on planning benchmarks.

LLM training jobs vary systematically in how much throughput they keep when GPU power is cut, and a simple telemetry-derived index (PFI) can predict that flexibility online and steer power allocation so clusters shed power cheaply during grid peaks.

Frontier AI agents can optimize quantitative metrics in closed-loop scientific experimentation but their scientific reasoning deteriorates when system rules fall outside familiar biological priors, and sandboxes can measure this gap.

Failures in multi-agent LLM systems arise through cross-principal interactions that local safety checks miss, and the field lacks the execution-level evidence standards needed to prove an effect is genuinely multi-agent rather than inherited from single-agent vulnerabilities.

Spending extra inference-time compute on critique refinement and wrapping the target in a deployment-matched agent harness (DISH) significantly narrows the gap between simulated alignment audits and real deployments, making audits harder for target models to distinguish from real use.