On the Impact of Anonymization on the Performance of Large Language Models

2609.11335 ↗ · 2026-09-10 · cs.CL, cs.AI

Tobias Deußer, Max Hahnbück, Lorenz Sparrenberg, Tobias Uelwer, Christian Bauckhage, Rafet Sifa

The finding

Headline claim

Anonymizing PII in LLM inputs generally degrades performance, but the cost is highly model- and task-dependent: stronger models lose more, retrieval-augmented tasks collapse, reversible pseudonymization beats redaction, and telling the model its input was anonymized does not help.

The test

Five LLMs (GPT-4o mini, Teuken-7B, Llama-3.1-8B, Gemma-2-27B, Qwen2.5-72B) evaluated on eleven benchmarks (ARC, BBH, EQ-Bench, HellaSwag, IFEval, MedQA, MMLU-PRO, MUSR, RGB, TruthfulQA, WMT14 En-De), 6,210 sampled items per condition, with an LLM-based pseudonymizer (F1 >88%, recall >91%), temperature 0, plus follow-up comparisons of five anonymization techniques and two explicit anonymization prompts on GPT-4o mini for MedQA, MUSR, and RGB.

Effect size & scope

Drops are usually small (ARC, HellaSwag, IFEval under ~5 percentage points) but RGB falls 30 to 60 points for most models, while TruthfulQA consistently improves under anonymization for all five models; the strongest baselines (Qwen2.5-72B, GPT-4o mini) show the largest absolute drops; pseudonymization clearly outperforms redaction. Scope conditions: max 500 samples per benchmark (1,000 for multi-task sets), English-dominant benchmarks, coding and math excluded, no statistical significance tests reported in the provided text.

Limitations

The authors concede the anonymizer itself is imperfect (91% recall), sample counts are capped, coding and math domains were excluded because they contain little PII, and results are relative rather than state-of-the-art comparisons. Unstated: no variance or significance testing, a single anonymizer configuration defines the main condition, Teuken-7B scoring 0.00 on two benchmarks suggests format-handling confounds, and the TruthfulQA improvement is hypothesized (reduced entity-driven hallucination) rather than causally demonstrated.

Counterpoint

The headline claims about 'more capable models are more fragile' rest on only five models and could reflect benchmark format quirks or anonymizer errors rather than entity reliance; LLM-based anonymization is itself prompt-sensitive and its F1 is measured against its own judgments; sampling 500 items per benchmark leaves noise in smaller-effect findings; the 'stronger models lose more' pattern may just reflect that higher-baseline models have more room to fall; no confidence intervals are given.

Economy implication

Little or nothing yet at the macroeconomic level. The nearest concrete implication is for regulated-industry (health, legal, finance) deployments: privacy compliance pipelines that pseudonymize inputs will silently erode quality on retrieval-heavy workflows, and enterprises should budget for that utility loss and prefer reversible techniques, but the paper offers no cost, adoption, or revenue evidence.

Who this matters for

Regulatory & Legal: GDPR-style PII scrubbing before sending data to third-party model APIs has a quantified, uneven accuracy cost, so compliance design choices (reversible pseudonymization vs redaction) directly change legal exposure and output quality.

Technical Capability: The finding that stronger models rely more on specific entity information, and that anonymization can reduce hallucinated entity associations on TruthfulQA, is a concrete data point on what frontier capability is built from.

Societal & Cultural: For anyone building privacy-preserving AI in sensitive domains, the paper shows privacy and utility are not a uniform trade-off and can be co-designed rather than simply traded.

Abstract

As large language models are increasingly deployed in sensitive domains, anonymizing input data to protect personally identifiable information has become a critical practice. However, the impact of this anonymization on model utility is not well understood. This paper presents a systematic empirical study of the trade-off between privacy and performance. We evaluate five prominent language models across eleven diverse benchmarks, comparing their performance on original versus pseudonymized inputs. Our results reveal that while anonymization generally degrades performance, the effect is highly nuanced. We find that more capable models, such as Qwen2.5-72B and GPT-4o mini, suffer the largest performance drops, suggesting a stronger reliance on specific entity information. The impact is also task-dependent: performance on TruthfulQA improves with anonymization, while retrieval-focused tasks like RGB experience a catastrophic decline. Further experiments show that reversible anonymization techniques that preserve entity uniqueness significantly outperform irreversible ones like redaction, and that explicitly prompting models about anonymization offers no discernible benefit. We conclude that anonymization is not a one-size-fits-all solution and must be co-designed with the model and task in mind to balance privacy and utility effectively. Our findings provide a crucial baseline for developing more robust, privacy-aware AI systems.