Overview
The study uses multiple models, established trustbenchmarks, and several compression tools with repeated seeds; results are empirical and reproducible but limited to training-free compression on 13B-class models and depend on calibration data and model choice.
Citations4
Evidence Strength0.75
Confidence0.78
Risk Signals11
Trust Signals
Findings with numeric evidence: 6/6
Findings with evidence refs: 6/6
Results with explicit delta: 4/5
Reproducibility
Status: Code + data available
Open source: Partial
At A Glance
Cost impact: 80%
Production readiness: 70%
Novelty: 60%
Why It Matters For Business
Compression can save cost and enable deployment on consumer GPUs, but it can also change model safety in ways that accuracy tests miss. Pick compression methods and bit-rates with trust tests, not just MMLU.
Who Should Care
Summary TLDR
The paper systematically measures how popular, training-free compression techniques change an LLM's trustworthiness across eight dimensions (toxicity, fairness, ethics, privacy, adversarial/OOD robustness, stereotypes, robustness to adversarial demonstrations) plus standard utility (MMLU). Main takeaways: weight quantization (especially 4-bit, using AWQ/GPTQ) usually preserves benign accuracy and often preserves or improves some trust metrics; structured pruning (N:M, 50%) tends to degrade trust; extreme 3-bit quantization can cause large, unpredictable safety failures (GPTQ can break instruction-following and raise toxicity). The authors release code, models, and a modified DecodingTrust/ M
Problem Statement
We lack a comprehensive, multi-dimension view of how compression changes LLM trust. Developers compress large models to save compute, but most evaluations only check accuracy or perplexity. This leaves safety risks (toxicity, privacy, fairness, robustness, ethics, etc.) unmeasured and possibly hidden.
Main Contribution
A broad, systematic benchmark of compression effects on trustworthiness: 3 popular 13B models, 5 training-free compression methods, and 8 trust metrics (DecodingTrust) plus MMLU.
Empirical finding that quantization (4-bit, AWQ/GPTQ) often preserves utility and can improve some trust metrics, while pruning at hardware-friendly patterns (2:4) often harms trust.
Key Findings
4-bit post-training quantization usually preserves trustworthiness within small margins.
Quantization can sometimes improve ethics or fairness.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| Trustworthiness preservation at 4-bit | ≤5-point average drop across 8 trust metrics (LLAMA2 13b Chat, 4-bit) | 13b dense | — | DecodingTrust | Fig.3, Sec.5.1 | Figure 3 |
| Ethics improvement (example) | 54.1 → 76.3 (GPTQ 4-bit for LLAMA2 13b Chat) | 13b dense Ethics score 54.1 | +22.2 points | DecodingTrust - Ethics | Sec.5.1 (Case Study 1) | Figure 3 and text |
What To Try In 7 Days
Quantize a production 13B model to 4-bit (AWQ) and run your safety suite (toxicity, privacy, fairness, OOD).
Avoid one-shot structured pruning at 2:4 without per-metric validation; compare trust metrics before/after.
If using GPTQ, run compression with multiple calibration seeds and measure variance in trust scores.
Optimization Features
Infra Optimization
Model Optimization
System Optimization
Training Optimization
Inference Optimization
Reproducibility
Code URLs
Data URLs
Risks & Boundaries
Limitations
Focus on training-free, post-training compression (GPTQ, AWQ, SparseGPT, Wanda); does not evaluate distillation or retraining-based compression.
Most experiments center on 13B-class models; results may not generalize to much smaller or much larger models.
When Not To Use
When you require extreme compression (3-bit) in safety-sensitive systems without extensive trust testing.
If you must use aggressive structured pruning (50% N:M) without per-dimension validation.
Failure Modes
Loss of instruction-following after aggressive quantization, producing malformed or unsafe outputs.
Sharp increases in toxicity when using GPTQ at 3-bit due to low refusal rates.