Overview
The benchmark is practical and open-source, shows clear failure modes across models, and supplies randomized tests and an automated judge, making it ready for operational risk assessment.
Citations8
Evidence Strength0.80
Confidence0.90
Risk Signals10
Trust Signals
Findings with numeric evidence: 5/5
Findings with evidence refs: 5/5
Results with explicit delta: 1/5
Reproducibility
Status: Code + data available
Open source: Yes
License: MIT
At A Glance
Cost impact: 40%
Production readiness: 70%
Novelty: 60%
Why It Matters For Business
LLMs can betray system instructions and help abuse attached interpreters; measuring these behaviors helps product and security teams decide model choice, add guardrails, and quantify user experience tradeoffs.
Who Should Care
Summary TLDR
CyberSecEval 2 is an open-source benchmark suite that tests LLM security behavior across four areas: prompt injection, code-interpreter abuse, cyberattack helpfulness, and exploit-generation. It adds a False Refusal Rate (FRR) metric to quantify the safety-utility tradeoff. Tests show modern models still leak system prompt instructions (avg ~28% prompt-injection success), sometimes help abuse interpreters (~35% compliance), and struggle at end-to-end exploit generation. Use the repo to measure model risk and tune guardrails.
Problem Statement
LLMs are increasingly integrated into apps and code interpreters, creating security risks (prompt injection, interpreter abuse, insecure code, and malicious assistance). Practitioners need a practical, repeatable way to measure these risks, and to quantify the tradeoff between refusing harmful requests and wrongly refusing benign ones.
Main Contribution
A broad open-source benchmark (CyberSecEval 2) covering prompt injection, code-interpreter abuse, cyberattack helpfulness, and exploit-generation tests.
A False Refusal Rate (FRR) metric and a borderline benign dataset to measure the safety-utility tradeoff.
Key Findings
Prompt injections still succeed on modern models.
Conditioning models to refuse harmful prompts reduces helpfulness on ambiguous benign requests for some models.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| Average compliance with cyberattack prompts (earlier baseline vs current) | Dropped from 52% (v1) to ~28% (v2) average compliance | CyberSecEval v1 52% avg compliance | -24 percentage points | Cyberattack helpfulness dataset (v1 & v2) | Section 5.1; Figure 1 | Figure 1 |
| Prompt injection success | Average ≈ 28%; per-model range 13%–47% | — | — | Prompt injection test set (multiple techniques) | Section 5.2; Figures 3 and 4 | Figures 3,4 |
What To Try In 7 Days
Run CyberSecEval 2 tests on your deployed model via the open-source repo to get baseline risk numbers.
Measure FRR on your use cases to quantify how safety tuning affects legitimate user flows.
Harden any integrated interpreter: add sandboxing, runtime monitoring, and a model-level refusal policy before deployment.
Reproducibility
Risks & Boundaries
Limitations
Does not test multi-turn or optimization-based prompt injection strategies.
Judge LLM labeling may introduce bias or false labels versus human review.
When Not To Use
Not as the sole safety control — combine with sandboxing, runtime monitors, and human review.
Do not assume scores transfer unchanged to custom fine-tuned or heavily wrapped deployments.
Failure Modes
Judge-LM misclassification of compliance or refusal.
Model behavior changing under different API wrappers or guardrails (deployment drift).