Overview
Paper provides a full system and empirical numbers on multiple edge devices; surrogate accuracy and co-design gains are demonstrated but require nontrivial profiling and compute to reproduce.
Citations3
Evidence Strength0.70
Confidence0.80
Risk Signals11
Trust Signals
Findings with numeric evidence: 5/5
Findings with evidence refs: 5/5
Results with explicit delta: 5/6
Reproducibility
Status: Partial assets available
Open source: Unknown
At A Glance
Cost impact: 70%
Production readiness: 60%
Novelty: 60%
Why It Matters For Business
Co-designing model and device cuts operational energy and peak power by an order of magnitude while keeping or slightly improving accuracy — lowering battery drain, thermal limits and cloud costs for on-device NLP.
Who Should Care
Summary TLDR
This paper presents a practical pipeline to jointly search transformer architectures and edge devices to meet accuracy and hardware targets. Key pieces: FlexiBERT 2.0 (a very large, heterogeneous transformer design space), ProTran (profiles latency, energy, and peak power and trains lightweight surrogates), BOSHCODE/EdgeTran (Bayesian co-design search for model-device pairs), and GPTran (block-level grow-and-prune post-processing). On GLUE, the final model (ET*) is 2.8× smaller than BERT-Base and yields 0.8% higher GLUE score while running on a selected Apple M1 GPU gives ~15% lower latency, ~10× lower energy, and ~10.8× lower peak power versus the server GPU baseline (Nvidia A100). FlexiB E
Problem Statement
Large transformer models are impractical on low-power edge devices because they can exceed latency, energy, and peak power budgets. Existing NAS or pruning work often optimizes only parameters or FLOPs and ignores energy and peak power across diverse mobile hardware. Searching this huge joint space (models × devices) needs fast profiling and surrogate models plus a codesign search that balances accuracy with latency, energy, and peak power.
Main Contribution
FlexiBERT 2.0: an expanded heterogeneous transformer design space (reported size ~1.7×10^88) and faster, finer-grained weight-transfer.
ProTran: an active-learning profiler that measures and models latency, energy, and peak power across multiple edge platforms and trains surrogates for fast queries.
Key Findings
Final co-designed model (ET*) is 2.8× smaller than BERT-Base and improves GLUE by 0.8 percentage points
On the chosen edge device (Apple M1 GPU) ET* reduces latency, energy, and peak power compared to BERT-Base on an A100 server GPU
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| GLUE score | 80.4% | BERT-Base 79.6% | +0.8% | GLUE (multi-task) | ET* after EdgeTran + GPTran | Table VIII; Section V-F |
| Model size (parameters) | 39.6M | BERT-Base 110M | 2.8× smaller | — | ET* final model | Table VIII; abstract |
What To Try In 7 Days
Profile your target device with ProTran (few hours) to get latency/energy/power surrogates.
Run a constrained search in FlexiBERT 2.0 using EdgeTran to find a model-device pair for your latency and power budget.
Apply GPTran grow-and-prune on the converged model to squeeze extra savings in size and pretraining loss.
Optimization Features
Infra Optimization
Model Optimization
System Optimization
Training Optimization
Inference Optimization
Reproducibility
Data URLs
Risks & Boundaries
Limitations
FlexiBERT 2.0 pretraining and surrogate construction is expensive (reported ~100 GPU-days).
Surrogates assume fixed batch size; changing batch sizes needs retraining of predictors.
When Not To Use
When you cannot afford initial profiling and surrogate training (no GPU budget).
When the target device runs multiple dynamic workloads whose interference you cannot pre-profile.
Failure Modes
Surrogate model mismatch: predictors may mis-rank architectures on unseen device behaviors.
Weight transfer may underperform for large topology changes, requiring longer fine-tuning.