Overview
Paper introduces a practical evaluation lens (Agentic ROI) and a roadmap. Evidence mixes a small survey (n=34) and conceptual arguments, so ideas are actionable but need broader empirical validation.
Citations0
Evidence Strength0.50
Confidence0.80
Risk Signals11
Trust Signals
Findings with numeric evidence: 4/4
Findings with evidence refs: 4/4
Results with explicit delta: 0/3
Reproducibility
Status: No open assets linked
Open source: Unknown
At A Glance
Cost impact: 70%
Production readiness: 40%
Novelty: 40%
Why It Matters For Business
Measure agent value as Agentic ROI (quality + time saved per dollar) to decide where to deploy agents profitably and avoid wasting resources on low-ROI, high-cost integrations.
Who Should Care
Summary TLDR
This position paper argues that the real bottleneck for widespread LLM agent adoption is low Agentic ROI—the user-facing ratio of information gain and time savings to cost. The authors define Agentic ROI, demonstrate its use with a 34-person survey across five domains, and show high ROI in coding/research but low ROI in mass-market tasks like office work and e-commerce. They propose a zigzag roadmap: first "scale up" agents (sleep-time compute, multi-step reasoning, proactive interaction) to increase information gain and time savings, then "scale down" (memory retrieval, distillation, quantization, hardware-software co-optimization) to cut per-task cost. The paper is a strategic call to re-e
Problem Statement
LLM agents can technically automate many tasks, but many real-world uses deliver too little net benefit to users once time, prompting effort, verification, and cost are accounted for. The paper introduces Agentic ROI to measure whether deploying an agent actually improves users' utility compared to human or UI alternatives.
Main Contribution
Introduce Agentic ROI: a simple, actionable metric combining information gain, time savings, and monetary cost to evaluate agent usability.
Present a small empirical demonstration (n=34 survey) showing Agentic ROI correlates strongly with reported usability (r=0.95).
Key Findings
Reported agent usability across domains aligns tightly with computed Agentic ROI.
High Agentic ROI appears in coding and scientific research; low ROI in office work, e-commerce, and personal assistance.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| Survey sample size | 34 participants | — | — | Section 3.1 | 34 survey responses (14 AI practitioners, 20 end-users) | Section 3.1 |
| Correlation between Agentic ROI and reported usability | r = 0.95 | — | — | Figure 1b | Strong positive linear correlation reported | Section 3.1, Figure 1b |
What To Try In 7 Days
Run a small ROI audit: pick one high-T0 workflow, log T0 and T_agent, and collect user quality ratings.
Add simple proactive features (prefilled templates, intent inference) to cut interaction time and re-measure ROI.
Pilot sleep-time compute or cached retrieval for repetitive tasks to estimate cost savings.
Agent Features
Memory
Planning
Tool Use
Frameworks
Is Agentic
Yes
Architectures
Collaboration
Optimization Features
Token Efficiency
Infra Optimization
Model Optimization
System Optimization
Training Optimization
Inference Optimization
Reproducibility
Risks & Boundaries
Limitations
Small empirical sample (34 survey responses) limits generalizability.
Cost estimates per task are coarse and normalized heuristically.
When Not To Use
Short, single-step interactions where UI is faster (low T0 tasks).
Deterministic, repetitive processes best served by RPA or rule systems.
Failure Modes
Prompting and verification overhead can erase time savings, yielding negative ROI.
Agent hallucination or drift during long multi-step tasks causes extra verification.