Overview
The paper offers a practical, no-training prompt fix and a benchmark. Results are supported by human and GPT-4 evaluations, but INDUST covers common-sense cases and may miss domain-specific or long-tail failures.
Citations3
Evidence Strength0.80
Confidence0.82
Risk Signals8
Trust Signals
Findings with numeric evidence: 4/4
Findings with evidence refs: 4/4
Results with explicit delta: 3/3
Reproducibility
Status: Code + data available
Open source: Partial
At A Glance
Cost impact: 30%
Production readiness: 60%
Novelty: 60%
Why It Matters For Business
Products that reuse LLMs risk amplifying users' false assumptions; adding a brief critique prompt is a low-cost way to reduce misinformation and potential harm.
Who Should Care
Summary TLDR
The paper introduces INDUST, a dataset of 'inductive instructions' that embed false premises in three styles (fact-check, question-with-false-premise, creative-with-false-premise). Off-the-shelf LLMs often accept those premises and produce wrong or unsafe outputs, especially for creative prompts and multi‑premise items. The authors propose DUAL-CRITIQUE prompting: ask the model to (1) critique the user's instruction and (2) critique its own answer. A single-step dual-critique prompt (SDUAL-CRITIQUE) consistently raises 'Truthfulness' and 'Helpfulness' across models and settings without extra training. They also provide LINDUST, a larger set for fine-tuning, and show LoRA fine-tuning helps.
Problem Statement
Users sometimes give instructions that already contain false or dangerous assumptions. LLMs often accept these assumptions and generate misleading or harmful content. We need practical ways to measure this failure and make models refuse, correct, or safely revise such instructions.
Main Contribution
INDUST: a labeled benchmark of ~3,000 instruction-response pairs covering three inductive styles (FCI, QFP, CIFP) and single vs multi-premise cases.
A validation that strong LLMs (ChatGPT, davinci-003, ChatGLM2, BELLE) are vulnerable to inductive instructions, especially QFP and CIFP and multi-premise items.
Key Findings
LLMs often accept false premises and produce incorrect or unsafe outputs.
Adding evidence to an automatic GPT-4 evaluator improves alignment with human judgment.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| Truthfulness (w/ evidence automatic eval) | 86.67% accuracy | Vanilla GPT-4 80.33% accuracy | +6.34 pp | Validation set for automatic evaluation | w/ evidence protocol aligns best with human labels | Table 1 |
| ChatGPT Truthfulness on CIFP (zero-shot) | 90.00% with SDUAL-CRITIQUE | 22.53% with STANDARD prompting | +67.47 pp | INDUST test set, CIFP single-premise | Large absolute improvement from SDUAL-CRITIQUE | Table 2 (ChatGPT rows) |
What To Try In 7 Days
Run INDUST on your deployed model to measure acceptance of false premises.
Add a single-step dual-critique prefix asking the model to check the instruction and its answer.
Use GPT-4 + evidence as an automatic evaluator to monitor truthfulness cheaply at scale.
Optimization Features
Training Optimization
Reproducibility
Code URLs
Data URLs
Risks & Boundaries
Limitations
INDUST focuses on common-sense, stable facts and excludes domain-specific or time-sensitive knowledge.
Data and rewrites are largely constructed with Chinese-sourced rumors and LLM rewrites, which may bias examples.
When Not To Use
As sole defense for domain-specific safety (medical, legal) where expert validation is needed.
When instructions do not embed false premises (standard QA or creative tasks where no safety risk exists).
Failure Modes
Prompt sensitivity: models that benefit most from SDUAL can be more sensitive to prompt wording.
Multi-premise confusion: performance drops on inputs with multiple false assumptions.