Overview
Theory is simple and broadly applicable; experiments cover several mid-size LLMs and tasks. Practical gains are demonstrated for KV compression and pruning, but tuning/calibration is model-dependent.
Citations0
Evidence Strength0.70
Confidence0.80
Risk Signals10
Trust Signals
Findings with numeric evidence: 3/3
Findings with evidence refs: 3/3
Results with explicit delta: 3/4
Reproducibility
Status: Code + data available
Open source: Partial
At A Glance
Cost impact: 70%
Production readiness: 60%
Novelty: 60%
Why It Matters For Business
TAPPA gives a cheap, model-side signal (q-similarity) to decide which parts of a model and which cached tokens are compressible. That can cut memory and latency for long-context inference and allow more aggressive structured pruning with less accuracy loss.
Who Should Care
Summary TLDR
The paper gives a simple unifying idea: whether an attention head shows a stable pattern or unpredictable jumps depends on how similar its query vectors are over time. They formalize this with TAPPA and a q-similarity metric, prove when common patterns (re-access, sequential, seasonal, periodic-diagonals) arise, and show that q-similarity can guide practical tasks (KV cache compression and structured layer pruning) to get better compression with small or no accuracy loss.
Problem Statement
Prior work cataloged many attention-head patterns but lacked a single explanation and a practical, low-cost signal to pick which heads or layers are compressible.
Main Contribution
TAPPA: a time-series theory that links attention shapes to query temporal continuity (q-similarity) and RoPE channel behavior.
Mathematical conditions for three predictable attention patterns: re-access (sinks), sequential (diagonals), and seasonal/periodic patterns.
Key Findings
High q-similarity (smooth queries) predicts predictable attention heads; low q-similarity predicts retrieval-like, unpredictable heads.
Layer pruning guided by TAPPA (q-similarity + Block Influence) improves average task accuracy under the same pruning ratio.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| q-similarity (per-head average) | Llama-3.1 ≈ 0.80; Qwen2.5 ≈ 0.86 (on GSM8K/AIGC) | — | — | Section F.3 (GSM8K, AIGC) | Head-wise q-similarity heatmaps and reported averages | F.3; Figure 8 |
| Accuracy | Llama-3.1-8B avg 59.11 after TAPPA-guided pruning | ShortGPT avg 53.51 | +5.60 | Zero-shot classification on PIQA/HellaSwag/Winogrande/ARC-easy | Table 2; Section 6.2 | Table 2 |
What To Try In 7 Days
Compute per-layer q-similarity (cosine of recent queries) on your model with a small calibration set.
Replace uniform KV budget allocation with q-similarity-adjusted layer budgets and measure end-to-end latency and accuracy on a few LongBench-like queries.
Use q-similarity combined with an existing layer importance metric (e.g., Block Influence) as a lightweight pruning proxy and test a small pruning ratio (10–30%) on downstream task
Optimization Features
Model Optimization
System Optimization
Inference Optimization
Reproducibility
Data URLs
Risks & Boundaries
Limitations
q-similarity is model- and layer-dependent; per-model calibration is recommended.
TAPPA focuses on predictable heads; unpredictable retrieval heads remain critical and are not compressible using this signal.
When Not To Use
When most heads have low q-similarity (retrieval-heavy models) — q-similarity won't identify compressible parts.
On models that do not use RoPE or use very different positional encodings without verifying the theory.
Failure Modes
Misclassifying retrieval heads as compressible leads to loss of critical context and degraded factuality.
Over-reliance on layer-average q-similarity may hide important per-head variability.