Overview
The idea is practical: build a short solver pipeline via a small validation set, use a consistency metric to stop, and route to stronger solvers only when needed; experiments on public datasets back the cost and accuracy claims.
Citations1
Evidence Strength0.80
Confidence0.80
Risk Signals9
Trust Signals
Findings with numeric evidence: 5/5
Findings with evidence refs: 5/5
Results with explicit delta: 3/5
Reproducibility
Status: Code + data available
Open source: Yes
At A Glance
Cost impact: 80%
Production readiness: 70%
Novelty: 60%
Why It Matters For Business
Adaptive per-question solver selection cuts cloud API bills and lets teams trade a small latency increase for big cost or accuracy gains on reasoning workloads.
Who Should Care
Summary TLDR
This paper introduces Adaptive-Solver (AS), a runtime framework that inspects an LLM's answer quality and, when needed, adapts the solving strategy (model, sample size, prompt, decomposition granularity). On eight reasoning datasets the method cuts API cost vs always-using GPT-4 by 46–85% while matching GPT-4 accuracy, or it raises accuracy by ≈4.5% at equal cost. The system uses a small validation set to build a short pipeline of solvers and a consistency-based stopping rule to avoid extra calls.
Problem Statement
Most LLM reasoning systems use one fixed solver (same model, prompt, sample size, decomposition) for all problems. That wastes budget on easy items and under-solves hard ones. The paper asks: can we dynamically allocate test-time compute—switching models, sample sizes, prompts, and decomposition—per question to reduce cost and improve accuracy?
Main Contribution
Adaptive-Solver (AS) framework: runtime evaluation + adaptation that selects solvers per question.
Four concrete adaptation levers: model routing, sample-size scheduling, prompting-method switching, and decomposition-granularity control.
Key Findings
The Adaptive-Solver can cut API costs by a large margin while keeping GPT-4-level accuracy.
At the same cost budget, AS improves accuracy over static baselines.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| API cost reduction vs GPT-4 | 46%–85% lower API cost | Always use GPT-4 | — | evaluated across 8 datasets | Abstract; Sec 5.2; Fig 6 | Abstract; Sec 5.2; Fig 6 |
| Accuracy | 92.49% (AS-MSPD) | G3.5-10-ZeroCoT 87.93% | +4.56% | GSM8K test | Sec 5.2; Fig 6 | Sec 5.2; Fig 6 |
What To Try In 7 Days
Run a small validation set (50–200 samples) and cache multiple responses per (model,prompt) pair.
Implement a consistency-based stopping rule (self-consistency) and tune thresholds to your budget.
Build a 2–4 step pipeline: cheap model + small sample → medium prompt/sample → strong model fallback and measure cost/accuracy trade-offs.
Agent Features
Memory
Planning
Tool Use
Frameworks
Architectures
Optimization Features
Token Efficiency
Model Optimization
System Optimization
Inference Optimization
Reproducibility
Risks & Boundaries
Limitations
Requires a validation set and pre-saved responses to configure pipelines.
Pipeline search needs many cached responses initially; configuration has nontrivial setup cost.
When Not To Use
When strict single-call low latency is required (real-time single-turn APIs).
If you cannot call multiple models or lack access to cheaper model variants.
Failure Modes
Mis-calibrated consistency thresholds cause premature stops or unnecessary fallbacks.
Pipeline overfits the validation set and underperforms on different test distributions.