Overview
Production Readiness
0.7
Novelty Score
0.6
Cost Impact Score
0.8
Citation Count
0
Why It Matters For Business
PMPO cuts evaluation cost by using log‑probabilities instead of sampling and external judges, enabling faster prompt tuning for deployed models and improving midsize model outputs without fine‑tuning.
Summary TLDR
PMPO is a prompt-optimization method that uses token-level cross-entropy (model log‑likelihoods) to score and pick prompt variants. It finds low-quality prompt segments via a mask-guided analysis, asks a model to rewrite those parts for hard examples, and selects the best prompts by minimizing loss in a single forward pass. This removes costly generation and external judges, works with small and large open models that expose log-probs, and shows consistent accuracy and alignment gains on BBH, GSM8K, AQUA‑RAT and AlpacaEval 2.0. Main caveat: PMPO needs access to token-level probabilities and can overfit in extremely low-data settings.
Problem Statement
Automatic prompt optimization today often scores candidate prompts by generating full outputs and using human judges or model self-evaluation. That is slow, costly, and unreliable for small models. We need a unified, efficient method that works across supervised and preference tasks and on smaller models without heavy generation or external scorers.
Main Contribution
PMPO: a unified framework that uses token-level cross-entropy as a lightweight evaluation signal to rank prompt variants without output sampling.
Mask-guided importance analysis to localize prompt spans that hurt performance, guiding focused edits.
Model-in-the-loop generation of prompt rewrites for high-loss (hard) examples and single-pass loss-based selection of the best variant.
Empirical demonstration across model sizes and tasks (BBH, GSM8K, AQUA‑RAT, AlpacaEval 2.0), with ablations showing each module’s contribution.
Key Findings
PMPO attains the highest average accuracy on BBH in the 1‑shot Qwen2.5‑14B experiments.
PMPO leads math benchmarks (GSM8K, AQUA‑RAT) in reported accuracy.
On AlpacaEval 2.0, PMPO increased Qwen2.5‑14B average win rate from 31.81% to 51.52% (automatic GPT‑4 Turbo judge).
PMPO improves intermediate reasoning quality as measured by a Process Reward Model.
Ablations show each PMPO module contributes: dropping TIM, BCA, and PrefLoss reduces BBH accuracy from 80.63% to 76.74%.
Results
Accuracy
Accuracy
Accuracy
AlpacaEval 2.0 win rate (Qwen2.5‑14B)
Process reward (step quality) on GSM8K
Accuracy
Who Should Care
What To Try In 7 Days
Run PMPO on a small held-out set (k=3 hard cases, n=4 variants, up to 20 iterations) using an open model that exposes token log‑probs (e.g., vLLM/Qwen).
Pair PMPO with few‑shot examples: optimize prompts first, then add 3–5 examples to test additive gains.
If using proprietary APIs, prototype per-token likelihood estimation on a tiny task to measure token cost before scaling.
Optimization Features
Token Efficiency
- Avoids autoregressive decoding during evaluation to reduce token consumption
- Rewriting still samples variants (generation cost) but selection is cheap
Inference Optimization
- Rank prompts by token-level cross-entropy in a single forward pass
- Batchable loss evaluation to score many variants cheaply
Reproducibility
Data Urls
- BBH, GSM8K, AQUA-RAT, AlpacaEval 2.0 (public benchmarks referenced; paper uses public splits)
Data Available
Open Source Status
- partial
Risks & Boundaries
Limitations
- Requires access to token-level log-probabilities; many commercial APIs do not expose full-sequence likelihoods.
- Not practical to run directly on black‑box APIs without expensive token-by-token calls (high latency and token cost).
- Can overfit in extremely low-resource setups (e.g., single example training).
- Prompt transfer from large to small models can degrade performance; prompts often work best on the originating model.
When Not To Use
- When you only have access to closed APIs that do not return per-token likelihoods.
- When you have only one labeled example and cannot risk prompt overfitting.
- When generator sampling cost is the dominant constraint and you cannot afford any generation for rewrites.
Failure Modes
- No guaranteed improvement every iteration; optimization may stall or keep the original prompt.
- Overfitting to training examples if dataset is too small or not diverse.
- Prompts optimized on one model can be suboptimal or harmful on much smaller models.
Core Entities
Models
- Qwen2.5 (0.5B, 14B, 32B)
- Qwen2.5-7B
- LLaMA3.1-8B
- Qwen2.5-Math-PRM-7B
- DeepSeek-R1-DistillQwen (1.5B)
Metrics
- Accuracy
- token-level cross-entropy loss
- win rate (AlpacaEval/GPT-4 Turbo judge)
- process reward score
Datasets
- BBH (BIG-Bench Hard)
- GSM8K
- AQUA-RAT
- AlpacaEval 2.0
Benchmarks
- BBH
- GSM8K
- AQUA-RAT
- AlpacaEval 2.0