Xinyu Fu, Narayan Ramasubbu, Dennis Galletta
Human error detection in LLM output improves when users first generate their own explanations of LLM errors, and stays higher over repeated use when later cued to that reasoning, because the problem is often retrieval failure rather than lack of knowledge or effort.
Two randomized lab-in-the-field experiments with 640 customer-facing employees at a large enterprise-software company in China: Study 1 (N=400) randomizes whether participants self-generate an explanation of LLM errors before receiving a standardized one; Study 2 (N=240) runs over a baseline day plus seven workdays and randomizes a daily message that either carries a self-chosen cue tied to earlier reasoning or a verification-neutral personalized phrase.
Self-explanation produced roughly a 10-percentage-point improvement in subsequent error detection plus more elaborated recall in Study 1. In Study 2, error detection declined over repeated use in both arms, but declined significantly more slowly with the self-chosen retrieval cue, supported by cue-evocation and cue-episode association measures. Effects are measured on one organization's employees reviewing one deployment context; generalization beyond this setting is untested.
The full text provided does not state author concessions beyond the note that oversight may also depend on organizational conditions (system transparency, user authority to intervene) which are held outside the study scope. Obvious unstated limits: a single company in a single country, one task domain, short follow-up (seven days), detection measured on seeded errors whose patterns were taught, no evidence on downstream decision quality or on whether effects persist past a week, and no cost data on the interventions.
The error patterns were taught immediately before testing, so the 10-point gain may reflect ordinary recency rather than a retrieval mechanism; seven workdays is a short decay window and the cue effect could be novelty or a demand effect from a personalized daily message rather than genuine reactivation of encoded reasoning; lab-in-the-field employees paid to participate may scrutinize more than real users; and the Moses-illusion framing suggests shallow semantic reading, which engagement or interface fixes could address as cheaply as self-explanation plus cues.
In the next 12 to 24 months the practical takeaway is that the human-review layer of enterprise AI deployments is a measurable quality bottleneck that cheap, lightweight interventions (self-explanation onboarding, personalized retrieval cues) can partially mitigate, which matters for firms selling human-in-the-loop workflows and for regulators drafting AI-oversight requirements. But this paper alone provides little or nothing yet on ROI, headcount effects, or vendor economics.
Labor & Knowledge Work: For firms staffing knowledge-work roles around LLM output, this shows the reviewer's effectiveness depends on memory and context, not just competence or effort, so onboarding and daily workflow design, not more hiring, may be the lever.
Regulatory & Legal: Regulators mandating human oversight of AI should note that oversight can fail even among capable, motivated employees, meaning compliance-by-attestation may overstate the real safeguard unless retrieval-supporting practices are in place.
Technical Capability: For teams evaluating LLM deployments, the paper reframes error-detection failures as partly a memory-retrieval problem, implying that even strong models need matched review-context design to keep humans useful.