Overview
Production Readiness
0.6
Novelty Score
0.6
Cost Impact Score
0.45
Citation Count
1
Why It Matters For Business
RAG pipelines used in products can be quietly manipulated by injected texts that bypass retrievers, filters, or LLMs; this risks wrong answers, hidden ads, or unwarranted refusals—test and harden the whole pipeline, not only the model.
Summary TLDR
SafeRAG is a Chinese benchmark and dataset that tests RAG (retriever → filter → generator) pipelines against four data-injection attacks: silver noise (partial answers), inter-context conflict, soft ad (implicit toxic/advertising text), and white DoS (false safety-warning refusal). The authors build a 100-question news-based base dataset, craft attacks primarily by human annotators with LLM help, and run systematic experiments across 14 retrievers/filters/generators. Results show RAG systems are broadly vulnerable: attacks are most effective when injected into filtered or retrieved contexts, some compressors or filters worsen results, and certain retrievers (Hybrid-Rerank) and generators (Ba
Problem Statement
Retrieval-augmented generation (RAG) systems combine external documents with LLMs, but attacker-supplied texts can be injected at indexing, retrieval, or filtering and mislead the system. Existing benchmarks use attack types that are often blocked by retrievers or simple filters, so we lack a realistic evaluation that exposes subtle, bypassing attacks across the full RAG pipeline.
Main Contribution
Define four improved attack types against RAG: silver noise, inter-context conflict, soft ad, and white DoS.
Construct SafeRAG: a mainly human-curated Chinese dataset built from 100 news Q–context pairs and targeted attack texts.
Provide an evaluation framework and attack-specific metrics (Retrieval Accuracy, F1 variants, ASR/AFR) and test 14 representative retrievers, filters, and generators.
Show practical weaknesses across retriever, filter, and generator components and publish code on GitHub.
Key Findings
RAG systems are vulnerable to subtle injection attacks (noise, conflict, toxicity, DoS) at multiple pipeline stages.
Attack effectiveness is highest when injected into filtered or retrieved contexts rather than the raw knowledge base.
Hybrid rerankers are more robust to silver-noise than DPR; retriever robustness ranking is HybridRerank > Hybrid > BM25 > DPR.
Some compressors/filters can hurt safety by removing needed details or failing to block attacks.
Generator choice matters: Baichuan 13B performs well on DoS in these tests; stronger models can be more sensitive to some attacks.
Results
Number of evaluated RAG components
Dataset size (base)
Baichuan-13B DoS robustness (F1_Variants)
Evaluator vs human consistency (selected metrics)
Who Should Care
What To Try In 7 Days
Run SafeRAG tests (or replay attacks) against your retriever+filter+generator stack.
Simulate silver-noise and white-DoS injections into filtered contexts to check end-to-end behavior.
Compare hybrid-rerank versus DPR/BM25 retrieval for your domain and prefer hybrid rerankers if noise is a problem.
Reproducibility
Code Available
Data Available
Open Source Status
- partial
Risks & Boundaries
Limitations
- Attack coverage focuses on data-injection attacks and does not include model-level backdoors.
- Benchmark targets single-modal text RAG (news); it does not evaluate multimodal or structured knowledge RAG.
When Not To Use
- Assessing model-level backdoors or attacks that modify model weights.
- Evaluating multimodal (image/audio/graph) RAG systems without adaptation.
- Measuring privacy leakage or side-channel/latency attacks.
Failure Modes
- Benchmark may reveal attack patterns that adversaries could adapt to (risk of method disclosure).
- Human-constructed conflicts are realistic but cannot cover all real-world manipulations or domains.
- Evaluations are on Chinese news data; transfer to other languages/domains may differ.
Core Entities
Models
- DeepSeek
- GPT-3.5-turbo
- GPT-4
- GPT-4o
- Qwen-7B
- Qwen-14B
- Baichuan-13B
- ChatGLM-6B
- DPR
- BM25
- Hybrid
- Hybrid-Rerank
- bge-base-zh-v1.5
- bge-reranker-base
Metrics
- Accuracy
- F1(correct)
- F1(incorrect)
- F1(avg)
- Attack Success Rate (ASR)
- Attack Failure Rate (AFR)
Datasets
- SafeRAG (Chinese RAG security evaluation dataset)
- base news Q–context dataset (100 pairs, Aug–Sep 2024)
Benchmarks
- RGB
- RAG Bench
- LRII
- RECALL
- Phantom
- MAR
- PoisonedRAG
- ClashEval