Compression cuts model memory, cost and inference latency so LLMs can run on fewer GPUs or at lower cloud cost; pick compression that fits your accuracy budget and hardware.

Compression makes large Transformers affordable to run and store: use post-training quantization and structured pruning for immediate cost and latency gains without full retraining.

Compression and better kernels let teams run large LLMs on fewer GPUs or even on single workstations, cutting hosting costs and enabling edge/embedded use cases without losing core capabilities.

If you run multiple model sizes, prune a big pretrained model and distill smaller variants to cut token and compute costs dramatically while keeping or improving accuracy.

HomoDistil produces smaller, better-performing BERT derivatives by pruning from the teacher while distilling; this saves storage and lowers fine-tuning costs while preserving quality—useful when you need compact models with higher accuracy than typical distilled alternatives.

You can shrink model footprint and inference cost while keeping high accuracy by distilling a Transformer into a smaller model, lowering hardware and energy bills for production NLP services.

AERA turns expensive LLM reasoning into a deployable, smaller model that scores answers and explains decisions, lowering inference costs and improving explainability for education products.

You can run a practical ViT-style segmenter on a $100–$150 Jetson Nano by combining distillation and fp16 quantization, giving near-teacher accuracy while keeping model size and RAM within real device limits.

You can shrink LLM-based recommenders to ~13% of original inference size and cut training/inference time by ~6–8× while keeping or slightly improving ranking quality, which reduces hardware cost and increases serving throughput.

You can offload planning to a cheap local model and keep expensive models for final solving, cutting inference cost while keeping accuracy on reasoning tasks.

DistilDP lets you produce a smaller, private language model with better utility than privately fine-tuning the small model directly, reducing deployment cost while respecting strong DP budgets.

Distillation cuts LLM inference cost and memory while keeping most capabilities; picking the right KD style (white‑box, black‑box, CoT, retrieval‑augmented) matters for accuracy, robustness, and deployment budget.

TSLD lets you quantize decoder LMs to 2-bit ternary weights with near full-precision quality and little extra training cost, reducing model size and inference memory while preserving reasoning accuracy.

If you run face recognition on mobile or edge devices, distilling a high-performing Transformer into a CNN can boost verification accuracy substantially while keeping hardware-friendly inference.

DSFormer reduces transformer model size substantially (2x–3.6x) while keeping accuracy close to original models and can be stacked with distillation/quantization to cut hosting or edge deployment costs further.

If you fine-tune assistants from pairwise preferences, distilling explicit reward models (and using small ensembles) reduces brittle failures from biased or sparse preference labels while keeping offline training simple.

You can run near-FM quality models in production by distilling then pruning; this cuts serving cost and latency so ranking and generative features scale to real traffic.

If you distill models from imperfect teachers, fixed offline distillation can degrade real-world quality; using online or more diverse data keeps smaller models reliable.

Smoothing labels with KD can reduce made-up facts in summaries and answers, improving trustworthiness in high-stakes apps while keeping core model accuracy.

Converting and then distilling long-context transformers cuts inference cost and latency substantially while keeping most accuracy, letting teams serve longer documents cheaper and on smaller hardware.

LSKD lets products accept user-pointed regions (tap/click) instead of long referring text, improving region-level answers and reducing UX friction for multimodal apps while using existing VL architectures.

BitDistiller makes deploying 2–3 bit LLMs practical: it keeps much of reasoning/code accuracy while slashing quantization time and GPU cost, enabling cheaper on-prem or edge inference.

You can make small-data multilingual models practical on constrained hardware: pruning and quantization cut size and latency substantially while keeping most accuracy, enabling on-device NER for African languages.

TI cuts fine-tuning compute and time for ultra-low-bit deployment, letting teams ship memory- and compute-cheaper models faster while retaining accuracy.