Measure and report when LLMs have seen benchmark data to avoid invalid NLP claims

Overview

Why It Matters For Business

If model evaluation is contaminated, product decisions and vendor comparisons can be wrong; verify exposure to benchmarks before basing choices on published scores.

Summary TLDR

This position paper argues that benchmark data contamination—when a model has seen test data during its training—threatens NLP evaluation. The authors define three contamination types (guideline, raw text, annotation), show contamination can occur at pretraining, fine-tuning and post-deployment steps, and propose practical detection measures: overlap search for open models and memorization/extractability tests for closed models. They call for a community registry, tooling, and review-time checks to flag compromised results.

Problem Statement

When a model has been trained on a benchmark's test data, reported performance is inflated and scientific claims can be wrong. Data exposure can come from many sources and is hard to detect, especially for closed models, so routine evaluations may be unreliable.

Main Contribution

Key Findings

Results

Who Should Care

What To Try In 7 Days

Reproducibility

Risks & Boundaries

Core Entities

Related Papers

• A weekly-updated, contamination-free medical benchmark plus automated rubrics that align better with physicians than LLM-as-a-judge
• When synthetic training data and LLM evaluators are related, evaluators unfairly favor the student models
• Auto-update benchmarks with two LLM-driven strategies to reduce leakage and tune difficulty
• Ko-H5 and an open Korean LLM leaderboard: private tests, new Korean tasks, and when benchmarks stop helping
• TreeEval: benchmark-free LLM evaluation via LLM examiner and tree planning