Overview
Results show consistent few-shot and modest data-rich gains on two public datasets and include ablations; however, experiments are limited to two domains and no released code reduces reproduction ease.
Citations0
Evidence Strength0.70
Confidence0.80
Risk Signals9
Trust Signals
Findings with numeric evidence: 3/3
Findings with evidence refs: 3/3
Results with explicit delta: 3/3
Reproducibility
Status: Partial assets available
Open source: Partial
At A Glance
Cost impact: 65%
Production readiness: 60%
Novelty: 60%
Why It Matters For Business
SAMPLE helps detect multimodal fake news with far fewer labels and far fewer trainable parameters than full fine-tuning, lowering data and compute costs for real-world monitoring systems.
Who Should Care
Summary TLDR
SAMPLE is a prompt-learning framework that fuses CLIP image-text features with RoBERTa-based prompts. It adaptively weights multimodal signals using standardized cosine similarity to reduce noisy, uncorrelated image-text pairs. On two benchmark datasets (PolitiFact, GossipCop) SAMPLE variants (discrete, continuous, mixed prompts) outperform standard RoBERTa fine-tuning, with the mixed prompt (M-SAMPLE) giving the largest gains, especially in few-shot settings. The method is lightweight (few trainable parameters) and practical for low-label regimes, though the soft verbalizer and uncorrelated modalities remain limitations.
Problem Statement
Fake news is hard to detect from text alone. Multimodal fusion helps but naïve fusion can add noise when image and text are weakly related. Fine-tuning large models needs lots of labeled data and updates many weights. The paper asks: can prompt learning plus a similarity-aware multimodal fusion (using CLIP) give strong fake-news detection with few labels and fewer trainable parameters?
Main Contribution
SAMPLE: a multimodal prompt-learning pipeline that combines discrete, continuous and mixed prompts with a soft verbalizer and RoBERTa.
A similarity-aware fusion step using CLIP features and standardized cosine similarity to scale multimodal intensity and reduce noisy cross-modal signals.
Key Findings
Mixed prompting (M-SAMPLE) gives clear few-shot gains over standard fine-tuning.
SAMPLE can substantially beat older multimodal models on some settings.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| Average F1 improvement (few-shot) | M-SAMPLE +0.05 vs FT-RoBERTa | FT-RoBERTa | +0.05 | Aggregate few-shot experiments | Section 4.4 average improvements reported | Section 4.4 |
| PolitiFact data-rich F1 | 0.80 | FT-RoBERTa 0.79 | +0.01 | PolitiFact (data-rich) | Table 2, M-SAMPLE data-rich F1 0.80 | Table 2 |
What To Try In 7 Days
Run a small test: extract CLIP features and apply mixed-prompt RoBERTa on a labeled sample (k=8) to compare F1 vs your fine-tuned model.
Implement the standardized cosine-similarity scaling when fusing image+text to see if it reduces noisy cross-modal cases.
Measure trainable parameters and training time: try prompt tuning to cut costs compared to full fine-tuning.
Optimization Features
Training Optimization
Reproducibility
Data URLs
Risks & Boundaries
Limitations
Soft verbalizer is hard to optimize with large vocabularies in very low-data regimes.
Similarity-aware fusion reduces noise but does not explicitly model or correct uncorrelated cross-modal relations.
When Not To Use
When you have abundant labeled data and can afford full fine-tuning: gains shrink in data-rich regimes.
When image and text are consistently missing or images are irrelevant: visual modality can hurt few-shot performance.
Failure Modes
Visual features can inject noise when image-text correlation is low, hurting few-shot F1.
Soft verbalizer may bias predictions if label-word coverage is incomplete.