Survey: Reframe LLMs as agents that plan, act, and continually learn

January 18, 20268 min

Overview

Decision SnapshotReady For Pilot

Useful practically as a system design map: it organizes patterns and trade-offs for planners, tool use, memory, and multi-agent pipelines, but it is a literature synthesis rather than new experimental proof—expect guidance rather than turnkey recipes.

Citations0

Evidence Strength0.75

Confidence0.78

Risk Signals10

Trust Signals

Findings with numeric evidence: 2/4

Findings with evidence refs: 4/4

Results with explicit delta: 1/1

Reproducibility

Status: Partial assets available

Open source: Partial

At A Glance

Cost impact: 70%

Production readiness: 60%

Novelty: 60%

Authors

Tianxin Wei, Ting-Wei Li, Zhining Liu, Xuying Ning, Ze Yang, Jiaru Zou, Zhichen Zeng, Ruizhong Qiu, Xiao Lin, Dongqi Fu, Zihao Li, Mengting Ai, Duo Zhou, Wenxuan Bao, Yunzhe Li, Gaotang Li, Cheng Qian, Yu Wang, Xiangru Tang, Yin Xiao, Liri Fang, Hui Liu, Xianfeng Tang, Yuji Zhang, Chi Wang, Jiaxuan You, Heng Ji, Hanghang Tong, Jingrui He

Links

Abstract / PDF / Code

Why It Matters For Business

Treating LLMs as agents unlocks real-world automation: models can call tools, check work, store experience, and coordinate—cutting manual glue code and raising automation value across search, coding, science, and operations.

Who Should Care

Summary TLDR

This survey defines "agentic reasoning": treating LLMs as agents that plan, act (call tools), and learn over time. It organizes work into three layers—foundational (planning, tool use, search), self-evolving (feedback and memory), and collective (multi-agent coordination)—and two optimization modes: in-context (no parameter change) and post-training (SFT/RL). The paper reviews representative systems, benchmarks, and real applications (math, science, robotics, healthcare, web agents), highlights practical patterns and system designs, and outlines open problems such as long-horizon credit assignment, world-models, scalable multi-agent training, personalization, and governance.

Problem Statement

LLMs perform well on closed, static reasoning tasks but fail to act, adapt, or improve in open-ended, dynamic environments. We need methods that turn passive text generation into iterative decision loops where models plan, call tools, use memory, integrate feedback, and coordinate across agents.

Main Contribution

Formalize "Agentic Reasoning" as three layers: foundational, self-evolving, collective.

Systematic review of methods for planning, tool use, search, memory, and feedback.

Key Findings

Agentic systems center reasoning as an explicit think-then-act loop.

Practical UseDesign agents to separate internal reasoning traces from external actions so you can search thoughts (plans) before committing tool calls.

Evidence RefSec. 2, Sec. 3 formalism

Large-scale supervised fine-tuning has been used to teach tool use at scale.

NumbersToolLLM trained on 16,000+ real-world APIs

Practical UseIf you need broad, dependable API calling, invest in SFT on structured API demonstrations but watch for overfitting to training patterns.

Evidence RefSec. 3.2.2 (ToolLLM description)

Results

MetricValueBaselineDeltaSplit / DatasetEvidenceEvidence Ref
Accuracy≈35% relative improvement (reported for MIRIX)35%multimodal QA (MIRIX report)Sec.5.3 Multi-agent memory (MIRIX reference)MIRIX [328]

What To Try In 7 Days

Prototype a plan-then-act prompt that separates 'thought' and 'action' for a common task.

Add a simple validator (e.g., unit test) to loop until a passing result for code generation.

Attach a small workflow memory (saved snippets) to reuse prior successful steps in repeated tasks.

Agent Features

Memory
flat retrieval memory (vector DB)structured memory (graphs, workflows)learned memory control (RL memory managers)
Planning
in-context planning (Tree-of-Thoughts, workflow prompts)post-training planning (RL, reward shaping)formal plan representations (code/PDDL)
Tool Use
in-context tool invocation (documented APIs)SFTRL-based multi-tool masteryorchestration frameworks (HuggingGPT, ToolPlanner)
Frameworks
LangChainLlamaIndexToolLLMHuggingGPTAutoAgentsAgent-Q
Is Agentic

Yes

Architectures
plan-then-act pipelinesplanner-executor (hierarchical)tree-search / MCTS-based planningrole-based multi-agent pipelines
Collaboration
role decomposition (manager/worker/critic)router-based agent selectiontopology optimization and pruning

Optimization Features

Token Efficiency
workflow compressionmemory summarization (sleep-time compute)tool documentation to reduce prompt size
Infra Optimization
use smaller model for tool selection (GEAR pattern)amortize expensive tool-makers vs cheap tool-users
Model Optimization
SFTGRPO
System Optimization
planner-executor decouplingtool orchestration with hierarchical plannersagent routing and graph pruning
Training Optimization
self-supervised API annotation (Toolformer)distillation on improved trajectoriespretraining with simulated tool dialogues
Inference Optimization
in-context search / Tree-of-Thoughtsverifier-guided reranking & self-consistencydelegating selection to smaller LMs for cost savings

Reproducibility

Code AvailableYes
Data AvailableNo
Open Source StatusPartial
LicenseUnknown

Risks & Boundaries

Limitations

Survey: synthesizes many works but does not provide unified empirical comparisons.

Benchmarks vary in scope and the field lacks standard long-horizon evaluation protocols.

When Not To Use

For simple one-shot classification or short prompt tasks where no external action is needed.

Where strict privacy/regulatory constraints forbid external tool calls and shared memories.

Failure Modes

Compounding errors in long-horizon plans leading to derailment.

Hallucinated tool calls or incorrect parameterization when tool docs are insufficient.

Core Entities

Models

ReActTree-of-ThoughtsReflexionVoyagerToolformerToolLLMWebGPTMemGPT

Metrics

task success / pass ratevalidator acceptance (unit tests)recall / retrieval fidelityAccuracy

Datasets

GSM8KMATHGSM-style math datasetsToolQAAPIBench

Benchmarks

ToolBench (various)WebArenaWebWalkerMemBench / LongMemEvalAgentBench / MultiAgentBench

Context Entities

Models

GorillaAgent-QDeepResearcherAlphaEvolveMem0

Metrics

API invocation correctnesscitation-grounded synthesis qualitymulti-agent coordination stability

Datasets

ToolAlpaca tool dialogues (3,938 instances)MTU-Instruct (54,798 dialogues)APIBench (1,645 APIs, 16,450 instruction-API pairs)

Benchmarks

ToolFlow (multi-step tool tasks)WebRL / WebAgent-R1Mind2Web 2PaperQA