Overview
Solid empirical coverage across vision, NLU, NLG and large LMs supports production use in many cases; tune learning rates and sample sizes per task.
Citations5
Evidence Strength0.90
Confidence0.88
Risk Signals7
Trust Signals
Findings with numeric evidence: 5/5
Findings with evidence refs: 5/5
Results with explicit delta: 5/5
Reproducibility
Status: Code + data available
Open source: Partial
At A Glance
Cost impact: 70%
Production readiness: 80%
Novelty: 60%
Why It Matters For Business
FlexRound lowers precision without heavy retraining, letting you run large models with INT8/INT4 weights and small calibration sets while keeping near-original accuracy.
Who Should Care
Summary TLDR
FlexRound is a new post-training quantization (PTQ) rounding rule that divides pre-trained weights by learnable per-weight scales and a shared grid size, then rounds. This element-wise division makes the learned scales respond to weight magnitude, letting important (large) weights move farther from nearest-grid constraints. Across vision, NLU, NLG and large LMs (LLaMA), FlexRound narrows the accuracy gap to full precision for INT8/INT4 settings and often beats AdaRound/AdaQuant in per-tensor PTQ with small calibration sets.
Problem Statement
Post-training quantization often uses fixed rounding or additive learned offsets and can hurt accuracy, especially in low-bit and per-tensor PTQ. Existing adaptive rounding schemes cannot jointly learn a global grid size and per-weight scaling while leveraging the pretrained weight magnitudes.
Main Contribution
Propose FlexRound, a dividing-based learnable rounding: quantize by dividing each weight by a learnable per-weight scale and a shared grid size, then round.
Show theoretically that gradients w.r.t. per-weight scales are proportional to pretrained weight values, so larger weights naturally get larger scale updates.
Key Findings
FlexRound keeps ImageNet accuracy close to full precision when quantizing weights to 4 bits.
FlexRound outperforms AdaRound on mixed weight+activation quantization for MobileNetV2.
Results
| Metric | Value | Baseline | Delta | Split / Dataset | Evidence | Evidence Ref |
|---|---|---|---|---|---|---|
| ImageNet Top-1 / Top-5 (weights only, 4-bit) | ResNet-50 75.95% / 92.68% (FlexRound) | Full-precision 76.63% / 93.04% | -0.68% Top-1 | ImageNet | Table 2, per-tensor 4-bit weights | Table 2 |
| ImageNet Top-1 (weights+activations, 4-bit) | MobileNetV2 66.66% (FlexRound in QDrop) | MobileNetV2 full-precision 72.62% | -5.96% Top-1 | ImageNet | Table 3, Q+FlexRound 4/4 | Table 3 |
What To Try In 7 Days
Re-run PTQ for a production ResNet or MobileNet: 1k calibration images + FlexRound 5k iterations.
Apply FlexRound (8-bit per-tensor weights, per-channel for LLMs) to a small transformer (BERT base) and compare GLUE scores with existing AdaRound results.
Quantize one large model (e.g., GPT/OPT/LLaMA) to 8-bit with block-wise reconstruction and validate zero/few-shot accuracy on a target task.
Optimization Features
Model Optimization
System Optimization
Training Optimization
Inference Optimization
Reproducibility
Risks & Boundaries
Limitations
Performance can depend on learning-rate tuning for s1 and per-weight scales; default rates sometimes underperform AdaRound on some tasks (paper notes tuning helps).
MobileNetV2 sometimes needs careful preprocessing (CLE/AHB) depending on the pre-trained checkpoint.
When Not To Use
If you have full access to data and can perform quantization-aware training (QAT), QAT may still yield smaller gaps for extreme low-bit targets.
When calibration data are extremely scarce (<32 samples), FlexRound accuracy degrades.
Failure Modes
If gradient signals for certain weights are near-zero, FlexRound's per-weight scales may not change and those weights remain constrained to nearest grids.
Naive combinations with additive schemes (AdaQuant+FlexRound) gave mixed or worse results in the paper.