Overview
Production Readiness
0.7
Novelty Score
0.7
Cost Impact Score
0.7
Citation Count
0
Why It Matters For Business
ScoreFlow reduces manual workflow design and optimization cost by using a continuous, score-aware finetuning loop; this lets smaller, cheaper generator models reach or exceed larger-model baselines while lowering API optimization bills.
Summary TLDR
ScoreFlow is a system that generates and optimizes multi-agent (workflow) code using gradient-based finetuning on preference data. It replaces discrete search with a continuous optimization loop (Score-DPO) that uses numeric evaluation scores to weight preference pairs. On six benchmarks across question answering, coding, and math, ScoreFlow reports an average solve rate of 85.3%, beating prior automated and manual workflow methods by 8.2% and reducing optimization costs versus a representative baseline (Aflow). The method makes small open-source generators (e.g., Llama-3.1-8B-Instruct) competitive with larger models when paired with a strong executor.
Problem Statement
Manually authored LLM agent workflows are brittle and expensive to design. Prior automated methods use discrete search or single static workflows and struggle with scalability, adaptability, and noisy evaluation feedback. ScoreFlow aims to generate per-task code-style workflows and optimize the generator using evaluation scores directly, improving convergence and per-task adaptivity.
Main Contribution
ScoreFlow: an automated framework that generates code-style multi-agent workflows and iteratively refines the generator with execution feedback.
Score-DPO: a variant of Direct Preference Optimization that weights preference samples by score gaps and injects numeric scores into the ranking objective.
Extensive evaluation on six datasets (QA, coding, math) showing an average +8.2% solve-rate improvement and lower optimization cost compared to Aflow.
Key Findings
ScoreFlow improves average solve rate across six benchmarks.
Score-DPO outperforms standard DPO, PPO, and supervised finetuning inside the ScoreFlow pipeline.
ScoreFlow cuts optimization cost relative to a representative automated baseline (Aflow).
Results
Average solve rate (six benchmarks)
Per-dataset example — HumanEval (pass@1 / solve rate)
Optimization cost (total) — GPT-4o-mini executor
Who Should Care
What To Try In 7 Days
Clone the ScoreFlow repo and run the included demo with Llama-3.1-8B and GPT-4o-mini executor.
Use k=8 workflow samples per task and f(x)=x, d(x,y)=(x-y)^3 as default Score-DPO settings.
Measure solve rate and API cost vs a simple Chain-of-Thought baseline on one task (e.g., HumanEval).
Agent Features
Memory
- no explicit long-term retrieval memory reported
- per-workflow operator state passed via code
Planning
- per-task adaptive workflow generation
- conditional control flow (if/loops) in workflows
Tool Use
- vLLM for inference
- LoRA
- GPT-4o-mini / GPT-4o / DeepSeek as executors
Frameworks
- ScoreFlow (this work)
- Aflow (comparison)
- ADAS (comparison)
Is Agentic
true
Architectures
- code-as-workflow
- operator-based agents (programmer, reviewer, ensemble, test)
Collaboration
- multi-agent coordination through assembled operators
- ensemble and reviewer patterns for collaboration
Optimization Features
Token Efficiency
- generator outputs compact code workflows (reduces repeated prompt length)
- no explicit context compression technique
Infra Optimization
- experiments run on 2x A6000 GPUs
- use of vLLM to speed inference
Model Optimization
- LoRA
System Optimization
- replace discrete search with gradient-based optimization for scalability
- iterative loop with k=8 samples and M=3 iterations (default)
Training Optimization
- Score-DPO: score-weighted preference finetuning
- upweighted sampling of high score-gap pairs (d(x,y)=(x-y)^3)
Inference Optimization
- use of smaller generator models to lower API costs
- executor temperature 0 for evaluation consistency
Reproducibility
Data Urls
- HumanEval
- MBPP
- GSM8K
- MATH
- DROP
- HotpotQA
Code Available
Data Available
Open Source Status
- partial
Risks & Boundaries
Limitations
- Performance depends on executor quality and the chosen judge model.
- Score-DPO requires numeric evaluation scores; noisy or biased scores slow convergence.
- Hyperparameters f(·) and d(·,·) (e.g., exponent α) need tuning; extreme values hurt generalization.
- Experiments focus on code-style workflows and may not transfer to non-code representations.
When Not To Use
- You lack a reliable automatic executor or judge to produce numeric scores.
- Cost or latency prohibits running multiple workflow executions per task (k=8).
- Your application forbids code-generation or executable workflows for safety/regulatory reasons.
Failure Modes
- Overfitting to judge/executor biases—generator optimizes for the judge, not true correctness.
- Excessive upweighting (α too large) discards useful but noisy pairs and reduces generalization.
- Generated workflows may still produce runtime errors if template/instruction constraints are violated.
- Limited iterations (M small) may leave the generator under-optimized for diverse datasets.
Core Entities
Models
- Llama-3.1-8B-Instruct
- Qwen2.5-7B-Instruct
- GPT-4o-mini
- GPT-4o
- DeepSeek-V3
- DeepSeek-coder
Metrics
- solve rate
- F1 score
- win rate
- API cost ($)
Datasets
- HumanEval
- MBPP
- GSM8K (test subset)
- MATH (difficulty 5 subset)
- DROP
- HotpotQA
Benchmarks
- coding (HumanEval, MBPP)
- math reasoning (GSM8K, MATH)
- question answering (DROP, HotpotQA)