Overview
Method is experimentally well-supported on open-source judges and benchmarks; it needs attacker access to modify a candidate and relies on shadow-response simulation, so real-world risk is high for similar open systems.
Citations2
Evidence Strength0.80
Confidence0.88
Risk Signals11
Trust Signals
Findings with numeric evidence: 6/6
Findings with evidence refs: 6/6
Results with explicit delta: 1/6
Reproducibility
Status: Code + data available
Open source: Partial
At A Glance
Cost impact: 60%
Production readiness: 20%
Novelty: 70%
Why It Matters For Business
If your product uses LLMs to rank or judge content, attackers can bottle-manufacture short token suffixes that make the judge pick malicious or low-quality content. This can poison leaderboards, search results, automated labels for training, or tool selection.
Who Should Care
Summary TLDR
The paper introduces JudgeDeceiver, an automatic, gradient-guided method that appends a short injected token sequence to an attacker-controlled candidate. On open-source judge models and two evaluation sets, the attack forces the judge to pick the attacker’s response with very high success (often >90%) and remains robust to response-order changes. Common defenses (known-answer checks, perplexity filters) miss many attacks. The authors release code and evaluate transferability, ablations, and three real scenarios: LLM-powered search, RLAIF, and tool selection.
Problem Statement
LLM-as-a-Judge systems pick the best answer from multiple candidates. If an attacker can add text to one candidate, can they reliably bias the judge to choose that candidate? Existing prompt-injection and jailbreak tricks are manual and brittle. The paper asks whether an optimization-based injected sequence can consistently manipulate judge outputs across unknown candidate sets and positions.
Main Contribution
JudgeDeceiver: a first optimization-based attack that automatically generates injected sequences to bias LLM-as-a-Judge.
A loss formulation combining target-aligned generation, positional (target-enhancement), and adversarial perplexity terms, solved with discrete gradient-guided search.
Key Findings
JudgeDeceiver yields high attack success rates against open-source judges.
The attack keeps working when response order changes.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| ASR (attack success rate) | 90.8% average (Mistral-7B, MT-Bench) | — | — | MT-Bench | Table 1 (MT-Bench, Mistral-7B results) | Table 1 |
| PAC (positional attack consistency) | 83.4% average (Mistral-7B, MT-Bench) | — | — | MT-Bench | Table 1 (PAC values) | Table 1 |
What To Try In 7 Days
Audit recent judge decisions for suspicious clustering of a single submitter across queries.
Add human spot-checks for leaderboard entries and search filters, prioritizing high-impact queries.
Limit or sanitize untrusted candidate content before passing to the judge (e.g., strip suspicious trailing tokens). note this is imperfect but reduces risk quickly.
Optimization Features
Token Efficiency
Reproducibility
Data URLs
Risks & Boundaries
Limitations
Assumes attacker can submit or modify one candidate response and knows the target question-response pair.
Evaluations focus on open-source judges; proprietary API-based judges may behave differently.
When Not To Use
When all candidate responses are fully curated and not editable by external users.
When the judge model is a closed proprietary LLM with unknown behavior and no public prompt template.
Failure Modes
Human review or manual audits detect and override malicious selections.
Aggressive input sanitization or truncation removes or neutralizes the injected suffix.