Xiang Zhang, Lovedeep Gondara, Jekaterina Novikova
In financial QA, a majority (60%) of frontier-model errors are highly consistent across meaning-preserving perturbations, so agreement-based (label-free) evaluation certifies most errors as reliable and cannot detect them.
Example-level joint consistency-accuracy measurement across 4 financial QA datasets (CFA L1, CFA L2, CMT, FinQA, 200 items each), 10 perturbation dimensions with 3 variants each, two frontier models (GPT-5.5, Claude Opus 4.5) and 6 generator-validator configurations, yielding 23,170 observations with gold labels.
5.24% of all observations fall in the high-consistency/incorrect quadrant; conditioned on being wrong, 60% of errors do (1,213 of 2,014), and 77.7% of those show complete answer collapse (identical wrong answer across variants). CBI is threshold-invariant (7.2% at t=0.50 to 5.05% at t=1.00), shared across model families (shared CBI items 3.54x the independence expectation, chi-square p=1.24e-65), and concentrates with difficulty (5.22% CFA L2 vs 3.80% CFA L1) and free-form format (7.49% FinQA vs 5.42% CMT). Consistency gating buys roughly a one-third error reduction while retaining 60% of error mass.
The authors concede the findings are demonstrated only within financial QA; generalization to legal, biomedical, or engineering domains is untested. Perturbations are generated and validated by other LLMs, so perturbation quality depends on generator/validator models (though the 98.8% validator pass rate and cross-configuration stability mitigate this). Unstated: 200 items per dataset is modest; the 'consistency as proxy' strawman may not match how careful label-free evaluation pipelines actually operate; negation failures may be a distinct mechanism (semantic-inversion blindness) conflated with deterministic wrong preferences.
The setup is partly self-referential: perturbations are LLM-generated, so 'consistent but inaccurate' could partly reflect correlated errors in the perturbation pipeline or gold-label edge cases in FinQA. Datasets are small (200 each) and finance-specific, where exam-style questions may produce unusually confident wrong answers. The core insight, that self-consistent errors exist and defeat ensembling, was already established by prior work (Tan et al., Ding, BFF-Bench); the contribution is quantifying quadrant mass, an incremental refinement. Vendors using human-labeled or verified evals are unaffected.
Little direct economic evidence, but one clear near-term implication: any enterprise or regulated-deployment pipeline that substitutes LLM-agreement signals for human labels in high-stakes domains carries a quantifiable undetectable-error floor, which raises the true cost of label-free evaluation and modestly strengthens the case that reliable domain AI still requires expert-verified ground truth. It does not move capability, cost, or labor claims on its own.
Technical Capability: Evaluation methodology teams should treat consistency, majority vote, and cross-model agreement as conditional proxies that leave a deterministic, ensembling-proof error mass, not as reliability certificates.
Regulatory & Legal: Regulated financial deployments that lean on agreement-based proxies because ground-truth labels cannot be shared inherit a quantified blind spot, which matters for model-risk-management standards.