Overview
The method is practical and reproducible with public models and PEFT, but results come from single runs on scientific datasets and require heavy compute for long contexts.
Citations2
Evidence Strength0.60
Confidence0.70
Risk Signals13
Trust Signals
Findings with numeric evidence: 4/6
Findings with evidence refs: 6/6
Results with explicit delta: 5/5
Reproducibility
Status: Partial assets available
Open source: Partial
At A Glance
Cost impact: 35%
Production readiness: 60%
Novelty: 55%
Why It Matters For Business
You can get reliable, extractive summaries from large LLMs with modest adapter tuning (LoRA) and modern attention tricks, but expect much higher compute costs for long contexts.
Who Should Care
Summary TLDR
EYEGLAXS shows that decoder-only LLMs (LLAMA2-7B, ChatGLM2-6B) can be fine-tuned for long-document extractive summarization using LoRA (low-rank adapters), rotary positional interpolation, and FlashAttention2. On PubMed and arXiv, LoRA-tuned LLMs match or slightly beat strong extractive baselines (ROUGE-1 ≈ 49–50, ROUGE-2 ≈ 21–25). Benefits come with large compute costs (12K context training epochs take ~32–53 hours). Results are promising but tested only on scientific papers and reported from single-run experiments.
Problem Statement
Extractive summarization of long documents is reliable but usually uses encoder-based models; decoder-only LLMs are underused because long contexts and full fine-tuning are costly. The paper asks: can we efficiently adapt LLMs to extractive summarization of long texts using parameter-efficient fine-tuning and attention / positional tweaks?
Main Contribution
EYEGLAXS: a practical pipeline that fine-tunes decoder-only LLMs for extractive summarization using LoRA, Rotary Positional Embeddings (RoPE) interpolation, and FlashAttention2.
Apply LoRA adapters to Q/K/V and output projections so only a small fraction of parameters are trained while backbone weights stay frozen.
Key Findings
LoRA fine-tuning substantially improves extractive performance vs frozen LLMs.
LoRA yields similar gains for a different LLM backbone.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| PubMed ROUGE-1 (LLAMA2-7B, 12K) | 50.34 | GoSum 49.83 | +0.51 | PubMed test (full) | Table 2 shows LLAMA2-7B (12K) R1=50.34 vs GoSum 49.83 | Table 2 |
| PubMed ROUGE-1 (ChatGLM2-6B, 12K) | 50.17 | GoSum 49.83 | +0.34 | PubMed test (full) | Table 2 lists ChatGLM2-6B (12K) R1=50.17 | Table 2 |
What To Try In 7 Days
Run a pilot: fine-tune a public LLM (LLAMA2-7B or ChatGLM2-6B) with LoRA on a small domain dataset and measure ROUGE.
Replace standard attention with FlashAttention2 to test longer context support on available GPUs.
Profile epoch runtime at 4K and 12K to estimate compute budget before scaling production training.
Optimization Features
Token Efficiency
Infra Optimization
Model Optimization
System Optimization
Training Optimization
Inference Optimization
Reproducibility
Risks & Boundaries
Limitations
Experiments run once per setting (single-run), so variance and robustness are unmeasured.
High training cost for long contexts (12K training epochs take tens of hours per epoch).
When Not To Use
You have tight GPU/time budgets and cannot afford long-context training.
You need guaranteed behavior in domains not similar to PubMed/arXiv without further validation.
Failure Modes
Position bias: model over-selects sentences near document start and end.
Performance variability unknown due to single-run reporting.