Argmin AIArgmin AI
About usOptimize your agent
RLHFBasic RAGAgentic RAGQuantizationSafety BenchmarksLLM-as-a-JudgeChain-of-ThoughtMulti-Agent SystemsFunction CallingEfficient InferenceHallucination Detection
LLM EvaluationHuman EvaluationFactual ConsistencyAutomatic Evaluation

Human judges prefer LLM summaries; reference summaries often contain more hallucinations.

September 18, 20237 min

Overview

Production Readiness

0.7

Novelty Score

0.4

Cost Impact Score

0.6

Citation Count

32

Authors

Xiao Pu, Mingqi Gao, Xiaojun Wan

Links

Abstract / PDF

Why It Matters For Business

Zero-shot LLMs can produce higher-quality, more factual summaries than many human references and fine-tuned models, so businesses can often deploy LLM summarization directly and shift effort to dataset curation and verification.

Summary TLDR

The authors built fresh, small evaluation sets (50 examples per task) and ran human pairwise comparisons across five summarization tasks. GPT-series LLMs (text-davinci-003, GPT-3.5, GPT-4) were consistently preferred over human-written references and fine-tuned models. GPT-4 produced fewer sentence-level hallucinations than human references on several tasks, and had lower rates of extrinsic hallucination (40% vs human 62% average). The paper argues that standard summarization research focused on squeezing metrics on old datasets needs rethinking; future work should target higher-quality test sets, application-driven tasks, and better evaluation.

Problem Statement

Do modern LLMs already match or beat human and fine-tuned summarizers on real summarization tasks? The paper tests zero-shot LLM generation across five tasks using newly collected post-cutoff data and human pairwise judgments to measure overall quality and factual consistency.

Main Contribution

New human evaluation datasets for five summarization tasks (50 samples each) constructed after common LLM training cutoffs.

Large-scale human pairwise comparison showing GPT-series LLMs are preferred over human references and fine-tuned models across tasks.

Manual annotation and analysis of sentence-level hallucinations showing humans often introduce more extrinsic hallucinations.

A short roadmap arguing for new, higher-quality datasets, application-oriented summarization, and better evaluation methods.

Key Findings

Human judges prefer LLM summaries over human-written and fine-tuned model summaries in pairwise comparisons.

NumbersHuman preference scores for LLMs exceed 50% across tasks (Figure 4).

GPT-4 produced fewer sentence-level hallucinations than human references on several tasks.

NumbersSentence-level hallucination counts (GPT-4 vs Human): single 8 vs 13; multi 5 vs 62; cross-lingual 16 vs 15; dialogue 5

Human-written summaries have a higher share of extrinsic hallucinations than GPT-4 on average.

NumbersExtrinsic hallucination rate: GPT-4 avg 40% vs Human avg 62%.

Annotation agreement is moderate, suggesting human judgments are useful but noisy.

NumbersInter-annotator Cohen's kappa = 0.558.

Results

Human preference (pairwise win rate)

ValueLLMs preferred over human and fine-tuned systems across tasks

Sentence-level hallucination counts (GPT-4)

Valuesingle 8, multi 5, cross-lingual 16, dialogue 5, code 9

BaselineHuman: single 13, multi 62, cross-lingual 15, dialogue 15, code 46

Extrinsic hallucination proportion

ValueGPT-4 avg 40% vs Human avg 62%

Inter-annotator agreement

ValueCohen's kappa = 0.558

Who Should Care

Product ManagerFounderMl EngineerData ScientistCeo

What To Try In 7 Days

Run zero-shot prompts on your summaries and compare against existing pipeline outputs using quick pairwise human checks.

Audit your reference summaries for extrinsic facts not present in sources and correct them.

Replace ROUGE-only checks with a small human evaluation focusing on factuality and usefulness.

Reproducibility

Open Source Status

  • unknown

Risks & Boundaries

Limitations

  • Each evaluation dataset contains only 50 samples per task, limiting statistical power.
  • Only GPT family models were tested; LLaMA/Vicuna excluded due to unknown training cutoffs.
  • Human annotation is moderately subjective (kappa = 0.558).
  • Hallucination annotations and some analyses focus on GPT-4 as the LLM proxy.

When Not To Use

  • Don't generalize results to LLMs with unknown data cutoff or different architectures.
  • Don't assume the same gains hold on very long documents or niche domains not covered by the five tasks.
  • Avoid relying solely on pairwise human preference if you need strict factual verification for high-stakes decisions.

Failure Modes

  • Reference summaries contain extrinsic facts and can mislead both training and evaluation.
  • Small evaluation sets may overstate LLM advantages on broader data distributions.
  • Annotator bias or low agreement can distort preference scores.

Core Entities

Models

  • GPT-3 (text-davinci-003)
  • GPT-3.5
  • GPT-4
  • BART
  • T5
  • Pegasus
  • MT5
  • MBART
  • Codet5

Metrics

  • Pairwise win rate (human preference)
  • Human preference score
  • Sentence-level hallucination counts
  • Proportion of extrinsic hallucinations
  • Cohen's kappa (0.558)

Datasets

  • New single-news (post-2021, 50 samples)
  • New multi-news (post-2021, 50 samples)
  • New dialogue (post-2021, 50 samples)
  • New cross-lingual (translated single-news, 50 samples)
  • New code (Go programs, 50 samples)