Argmin AIArgmin AI
About usOptimize your agent
RLHFBasic RAGPrompt OptimizationAgentic RAGQuantizationSafety BenchmarksLLM-as-a-JudgeChain-of-ThoughtMulti-Agent SystemsFunction CallingEfficient InferenceHallucination Detection

Small prompt or format changes can reorder LLM leaderboards by many ranks

LLM EvaluationBenchmark RobustnessBenchmark LeakageBenchmark ConstructionAnalysis
February 1, 20247 min

Overview

Decision SnapshotNeeds Validation

Solid empirical evidence across 11 models and two benchmarks shows real risk in relying on single MCQ leaderboards; results are reproducible via provided code but do not offer a complete fix.

Citations2

Evidence Strength0.90

Confidence0.85

Risk Signals8

Trust Signals

Findings with numeric evidence: 6/6

Findings with evidence refs: 6/6

Results with explicit delta: 5/6

Reproducibility

Status: Code + data available

Open source: Partial

At A Glance

Cost impact: 70%

Production readiness: 40%

Novelty: 50%

Authors

Norah Alzahrani, Hisham Abdullah Alyahya, Yazeed Alnumay, Sultan Alrashed, Shaykhah Alsubaie, Yusef Almushaykeh, Faisal Mirza, Nouf Alotaibi, Nora Altwairesh, Areeb Alowisheq, M Saiful Bari, Haidar Khan

Links

Abstract / PDF / Code / Data

Why It Matters For Business

If you pick models by single MCQ leaderboard snapshots you risk choosing a weaker or misfitted model; small eval details can change rank and therefore cost and product outcomes.

Who Should Care

CTOProduct ManagerML EngineerData ScientistFounder

Summary TLDR

Leaderboards built from multiple‑choice benchmarks are brittle. Small, innocuous changes—reordering choices, swapping the letter symbols, or changing scoring—can move models up or down many ranks on MMLU and ARC. The paper catalogs three classes of tiny perturbations (choice order/IDs, prompt/scoring, and in‑context examples), measures their effects across 11 models, and recommends hybrid scoring and cautious interpretation of MCQ leaderboards. Code is available.

Problem Statement

Practitioners use MCQ leaderboards to pick expensive LLMs. But small, implementation‑level choices in prompts and scoring can massively change leaderboard order, risking wrong model selection and wasted cost.

Main Contribution

Systematic study showing MCQ leaderboard rankings are highly sensitive to small perturbations.

Isolation of three perturbation classes: answer choice format/order, prompt/scoring, and in‑context example content.

Key Findings

Minor perturbations can shift model ranks by many positions on MMLU.

NumbersUp to 8 rank positions change; example Yi‑6b moved 3→9 (Table 1)

Practical UseDo not trust single leaderboard snapshots for model selection; validate with multiple prompt/formats before buying or deploying a model.

Evidence RefFigure 1; Table 1

Leaderboards often disagree under small changes (Kendall kτ falls below stability threshold).

Numberskτ dropped to 0.564 under choice shuffles (kτ ≤ 0.75 indicates major disagreement)

Practical UseReport ranking sensitivity (e.g., Kendall τ) alongside scores to show stability before comparing models.

Evidence RefSection 5.1; Table A.5; Table 1

Results

MetricValueBaselineDeltaSplit / DatasetEvidenceEvidence Ref
Max rank displacementUp to 8 positions (MMLU)——MMLUFigure 1; AbstractFigure 1; Table 1
Ranking agreement (Kendall kτ)kτ = 0.564 after random choice shuffleskτ = 1.0 (original)−0.436MMLU subsetTable 1; Section 5.1Table 1

What To Try In 7 Days

Re-evaluate candidate models using hybrid scoring and report kτ to show ranking stability

Run 3 quick perturbations (shuffle choices, swap option symbols, and cloze vs symbol) and compare ranks

Sanitize few‑shot/context examples and rerun tests to detect leakage before deployment

Reproducibility

Code AvailableYes
Data AvailableYes
Open Source StatusPartial
LicenseUnknown

Code URLs

https://github.com/National-Centerfor-AI-Saudi-Arabia/lm-evaluation-harnesshttps://arxiv.org/pdf/2402.01781v2

Data URLs

MMLU (Hendrycks et al., 2020)ARC-Challenge (Clark et al., 2018)

Risks & Boundaries

Limitations

Cannot quantify root causes of token/position bias because pretraining data for models is not available

Proposed mitigation (hybrid scoring) reduces bias but is not a full solution

When Not To Use

When evaluating non‑MCQ tasks like freeform generation or long‑form reasoning

When you require a definitive, deployment‑grade ranking without further validation

Failure Modes

Leaderboard rank swaps due to answer ID tokens or choice ordering

High apparent accuracy driven by leaked answers in few‑shot context

Core Entities

Models

phi-2Yi-6bYi-34bMistral-7bMistral-7b-InstructLlama-2-7bLlama-2-7b-chatLlama-2-13bLlama-2-13b-chatLlama-2-70bLlama-2-70b-chat

Metrics

AccuracyKendall's τ (kτ)RStd (recall standard deviation)

Datasets

MMLU (Massive Multitask Language Understanding)ARC-Challenge (ARC-C)

Benchmarks

MMLUARC-C

You May Also Want to Read

Use all LLMs as judges: a fast, democratic way to rank models that matches human preference

0.60
0.60
0.70
0

De-Arena lets teams benchmark many models cheaply and comparably to humans; it reduces cost and single-judge bias so product and platform teams can maintain fair leaderboards and pick models for deployment faster.

Key finding

De-Arena aligns closely with human judges (Chatbot Arena).

Numbers: Spearman ρ = 0.974 (Overall, 66 LLMs)

Judge with hidden states: use small models' internal vectors instead of prompting large LLMs

0.60
0.60
0.70
0

You can replace expensive LLM-judge pipelines with cheap probes on small open models to filter and evaluate data at far lower cost while keeping most practical value for downstream fine-tuning.

Key finding

Probing small-model hidden states improves evaluation F1 over prompt-based inference by a large margin.

Numbers: Average F1 increased by >20% on most tasks

DIBJUDGE: fine-tune judges to separate true quality signals from translation artifacts

0.60
0.60
0.40
0

Automated multilingual evaluation can systematically over-score machine-translated outputs, harming product QA and model selection—DIBJUDGE cuts that bias while improving accuracy, lowering the risk of deploying models tuned to translation artifacts.

Key finding

DIBJUDGE improves multilingual reward-model accuracy and sets a new open-weight SOTA on m-RewardBench.

Numbers: m-RewardBench: DIBJUDGE-Qwen3-8B 91.37 ±0.22 vs Qwen3-8B 86.12 ±0.52 (Table 1)

LLM judges favor 'new' and 'expert' labels but never admit it.

0.30
0.35
0.20
0

If you use LLMs to auto-evaluate content (summaries, answers, stories), simple labels like 'written recently' or 'by an expert' can shift scores substantially. Decisions based on such automated judgments can be biased without visible reasons.

Key finding

Recency labels cause consistent selection shifts toward 'New' responses.

Numbers: GPT-4o VSR +30% on ELI5; Gemini +16% on ELI5; GPT-4o +16% on LitBench; Gemini +4% on LitBench

Confuse the judge: a black-box method that labels LLM evaluations as high or low uncertainty

0.60
0.60
0.40
0

You can flag LLM-generated ratings that are likely correct and route uncertain ones to humans, reducing verification cost and raising trust in automated evaluations.

Key finding

Low-uncertainty labels mark more accurate LLM evaluations.

Numbers: TruthfulQA Llama-3-70B: Low 0.91 vs Baseline 0.78 (Table 1)