Overview
Production Readiness
0.55
Novelty Score
0.65
Cost Impact Score
0.7
Citation Count
0
Why It Matters For Business
Memory systems add real operational cost and fragility; pick architectures that actually need external memory and measure token/time overhead before production.
Summary TLDR
This survey organizes memory-augmented generation (MAG) into four structural classes, then measures where current systems fail in practice. Main problems: many benchmarks are already solvable inside large context windows, lexical metrics (e.g., F1) mis-rank abstractive systems, weaker open models corrupt structured writes, and structured memories add big latency and token costs. The paper proposes a Context Saturation Gap (Δ) test, recommends LLM-based semantic judging with multi-rubric checks, and calls for backbone-aware decoding plus asynchronous maintenance to make agentic memory practical.
Problem Statement
Memory-augmented agents promise long-term personalization and reasoning, but evaluation and system design lag behind. Benchmarks, metrics, model choice, and maintenance cost are often misaligned with real-world constraints, hiding when external memory actually helps and when it becomes an operational burden.
Main Contribution
A compact, structure-first taxonomy of agentic memory: Lightweight semantic, Entity-centric/personalized, Episodic/reflective, Structured/hierarchical.
Systematic empirical analysis of four failure axes: benchmark saturation, metric misalignment, backbone sensitivity, and maintenance/latency costs.
Practical protocols and diagnostics: Context Saturation Gap (Δ) test and LLM-as-a-judge robustness checks.
Operational guidance linking memory structure to deployment trade-offs (accuracy vs. latency vs. cost vs. reliability).
Key Findings
Lexical metrics (F1) can reverse practical rankings versus semantic judgment.
Open-weight backbones produce many more structured-format errors than strong APIs.
Hierarchical/OS-style memory can make interactive latency unacceptable.
Offline construction costs vary widely and can be large in tokens/time.
Many existing datasets will be solved by long-context LLMs without external memory.
LLM-as-a-judge yields robust relative rankings across grading rubrics.
Results
Lexical F1 vs Semantic Judge (ranking mismatch)
Format error rate by backbone
Interactive user-facing latency (per turn)
Offline index construction tokens and time
Benchmark volume examples (saturation risk)
Who Should Care
What To Try In 7 Days
Run a Context Saturation Gap (Δ) on your tasks: compare MAG vs full-context baseline.
Replace F1 with an LLM-based semantic judge and test ranking stability across 2–3 rubrics.
Measure format error rates on your target backbone; add JSON/schema validators or constrained decoding if high errors appear.
Agent Features
Memory
- Short-term buffers
- Token-level latent memory
- Episodic buffers and consolidated summaries
- Persistent entity records
Planning
- Policy-optimized memory management
- Utility-aware retrieval (intent-guided recall)
Tool Use
- Dense embedding retrieval
- LLM-driven consolidation
- Graph traversal for multi-hop reasoning
Frameworks
- Hybrid index (dense+sparse)
- Hierarchical memory OS
- Graph-structured memory
Is Agentic
true
Architectures
- Lightweight Semantic
- Entity-Centric and Personalized
- Episodic and Reflective
- Structured and Hierarchical
Collaboration
- Asynchronous maintenance streams
- Profile-enriched role-playing contexts
Optimization Features
Token Efficiency
- Prompt-driven compression (e.g., ACON style)
- Latent token memory to reduce context tokens
Infra Optimization
- Separate maintenance workers to avoid throughput collapse
- Index sharding and efficient embedding services
Model Optimization
- Constrained decoding to stabilize structured outputs
System Optimization
- Asynchronous write/consolidation pipelines
- Validation layers for structured writes
Training Optimization
- RL
- RL-optimized compression
Inference Optimization
- Top-k retrieval reduction
- Context folding to reduce generation cost
Reproducibility
Data Urls
- HotpotQA (public)
- LongMemEval (public)
- MemBench (public)
Data Available
Open Source Status
- unknown
Risks & Boundaries
Limitations
- Survey-level analysis uses selected representative systems; not an exhaustive empirical sweep.
- LLM-as-a-judge depends on judge choice and still requires multi-rubric calibration.
- Construction and maintenance costs reported for specific configs; numbers vary by infra and model choices.
- No public code bundle or reproducibility scripts provided with this manuscript.
When Not To Use
- Tasks where all relevant info fits inside the model's context window (Δ ≈ 0).
- Low-entity-diversity tasks with minimal temporal state to preserve.
- Ultra-low-latency interactive systems where operations must be <1s and structured maintenance would block.
- Environments where target backbone cannot reliably produce structured outputs and validators are infeasible.
Failure Modes
- Silent write corruption: malformed JSON or hallucinated keys during memory updates.
- Paraphrase penalty: correct abstractive answers receive low lexical scores.
- Throughput collapse: maintenance latency lags user interactions, making memory stale.
- Backbone-dependent collapse: structured architectures fail more on weaker open models.
Core Entities
Models
- LOCOMO
- AMem
- MemoryOS
- Nemori
- MAGMA
- SimpleMem
- gpt-4o-mini
- Qwen-2.5-3B
Metrics
- F1
- BLEU
- Semantic Judge Score (LLM-as-a-judge)
- Format Error Rate
- User-Facing Latency (T_read+T_gen)
- Construction Time (h)
- Index Tokens (k)
Datasets
- LoCoMo
- HotpotQA
- LongMemEval-S
- LongMemEval-M
- MemBench
Benchmarks
- LoCoMo
- LongMemEval
- MemBench
- HotpotQA
Context Entities
Models
- MemAgent
- MemSearcher
- ACON
- TokMem
- PAMU
- MemOrb
- EMu
- MemP
- MAGMA (paper cited and evaluated)
Metrics
- LLM Judge Rubrics (MAGMA, Nemori, SimpleMem prompts)
Datasets
- LoCoMo (cited dataset)
- LongMemEval (cited dataset)
- Membench (cited dataset)