Overview
Production Readiness
0.6
Novelty Score
0.6
Cost Impact Score
0.3
Citation Count
3
Why It Matters For Business
Products that reuse LLMs risk amplifying users' false assumptions; adding a brief critique prompt is a low-cost way to reduce misinformation and potential harm.
Summary TLDR
The paper introduces INDUST, a dataset of 'inductive instructions' that embed false premises in three styles (fact-check, question-with-false-premise, creative-with-false-premise). Off-the-shelf LLMs often accept those premises and produce wrong or unsafe outputs, especially for creative prompts and multi‑premise items. The authors propose DUAL-CRITIQUE prompting: ask the model to (1) critique the user's instruction and (2) critique its own answer. A single-step dual-critique prompt (SDUAL-CRITIQUE) consistently raises 'Truthfulness' and 'Helpfulness' across models and settings without extra training. They also provide LINDUST, a larger set for fine-tuning, and show LoRA fine-tuning helps.
Problem Statement
Users sometimes give instructions that already contain false or dangerous assumptions. LLMs often accept these assumptions and generate misleading or harmful content. We need practical ways to measure this failure and make models refuse, correct, or safely revise such instructions.
Main Contribution
INDUST: a labeled benchmark of ~3,000 instruction-response pairs covering three inductive styles (FCI, QFP, CIFP) and single vs multi-premise cases.
A validation that strong LLMs (ChatGPT, davinci-003, ChatGLM2, BELLE) are vulnerable to inductive instructions, especially QFP and CIFP and multi-premise items.
DUAL-CRITIQUE prompting (USER-CRITIQUE + SELF-CRITIQUE), with single-step (SDUAL) and multi-step (MDUAL) variants that improve truthfulness and helpfulness without model training.
LINDUST: a larger synthetic dataset for fine-tuning; and a LoRA fine-tune on BELLE that improves zero-shot robustness.
Key Findings
LLMs often accept false premises and produce incorrect or unsafe outputs.
Adding evidence to an automatic GPT-4 evaluator improves alignment with human judgment.
Single-step Dual-Critique prompting sharply raises truthfulness and helpfulness across models.
Two-step (MDUAL) can help but risks error propagation and extra cost.
Results
Truthfulness (w/ evidence automatic eval)
ChatGPT Truthfulness on CIFP (zero-shot)
Helpfulness (average score 0-2)
Who Should Care
What To Try In 7 Days
Run INDUST on your deployed model to measure acceptance of false premises.
Add a single-step dual-critique prefix asking the model to check the instruction and its answer.
Use GPT-4 + evidence as an automatic evaluator to monitor truthfulness cheaply at scale.
Optimization Features
Training Optimization
- LoRA
Reproducibility
Code Urls
Data Urls
Code Available
Data Available
Open Source Status
- partial
Risks & Boundaries
Limitations
- INDUST focuses on common-sense, stable facts and excludes domain-specific or time-sensitive knowledge.
- Data and rewrites are largely constructed with Chinese-sourced rumors and LLM rewrites, which may bias examples.
- MDUAL-CRITIQUE can propagate critique errors and is sensitive to prompt wording.
When Not To Use
- As sole defense for domain-specific safety (medical, legal) where expert validation is needed.
- When instructions do not embed false premises (standard QA or creative tasks where no safety risk exists).
Failure Modes
- Prompt sensitivity: models that benefit most from SDUAL can be more sensitive to prompt wording.
- Multi-premise confusion: performance drops on inputs with multiple false assumptions.
- MDUAL error propagation: incorrect USER-CRITIQUE analysis can derail the SELF-CRITIQUE stage.
Core Entities
Models
- ChatGPT
- text-davinci-003
- ChatGLM2
- BELLE-7B
- GPT-4
Metrics
- Truthfulness
- Helpfulness
Datasets
- INDUST
- LINDUST
- CHEF
Benchmarks
- INDUST