You can cut ~20% of an LLM's parameters to save memory and speed without losing most zero‑shot ability, and re-tune it in hours on one GPU with modest public data—much cheaper than retraining or distillation.

Jamba lets teams process much longer documents on standard GPUs while cutting memory needs and speeding up inference; that reduces infrastructure cost for long-document products and enables new features like huge-context summarization.

SqueezeLLM cuts model storage and single‑request latency by ~2× while keeping near‑FP16 quality, enabling cheaper and faster on‑prem or cloud inference for generative LLMs.

Atom can multiply token throughput per GPU and shrink KV-cache memory, lowering cloud GPU costs or increasing capacity without major task-accuracy loss.

QA-LoRA lets teams fine-tune large models on far fewer GPUs, produce merged low-bit models, and deploy faster, cheaper INT4 inference without the accuracy loss from post-quantization.

RPTQ reduces memory footprint on large language models by up to ~80%, enabling cheaper hosting, fewer GPUs per deployment, and longer context lengths for customers without retraining models.

ShortGPT offers a low-effort way to cut model size and inference cost by removing low-impact layers; it preserves most classification-style performance and stacks with quantization to save more compute.

RRTF provides a lower-cost, scalable way to improve code-generation correctness by using unit tests and stronger-model outputs as ranked supervision; this delivers higher-quality code models that are faster and cheaper to run after quantization.

LLM inference can cost hundreds to thousands of watts per deployed model instance; choosing GPU type, shard layout, and power caps materially changes operating costs and latency.

Shrinking KV cache at inference increases batch size and throughput on fixed-memory GPUs without retraining, cutting hosting cost or enabling longer contexts on the same hardware.

Cut KV cache memory 3–7× and preserve accuracy so you can serve much longer contexts on existing GPUs, reducing infrastructure cost or enabling new long-document features.

OWQ reduces model storage and keeps accuracy with only tiny runtime and storage overheads, enabling deployment of very large LLMs on fewer GPUs and cheaper hardware.

Switching activations to FP8 and weights to FP4 can cut memory and exploit H100 FP8 hardware while keeping model quality—good for deploying large LLMs on constrained inference servers.

CompactifAI can cut model storage and runtime costs, enabling on-prem or cheaper-cloud LLM deployment with modest accuracy trade-offs for many tasks.

VeRA slashes the bytes required per adapted model (10–100× less) so firms can store many personalized or task-specific adapters on the same GPU. That reduces serving costs, speeds model swap-in, and lowers storage and network bandwidth for model variants.

ZipLM cuts inference cost and risk: it produces many valid speedup targets in one run and guarantees measured speedups on target hardware, reducing GPU/CPU time and deployment surprises.

Route cheap queries to local or smaller models to cut cloud inference costs while keeping user-facing quality high; thresholds let operators trade cost vs quality on demand.

Bonsai makes structured LLM compression feasible on commodity GPUs, cutting memory needs and producing faster models so teams can reduce inference cost and enable on-device fine-tuning without enterprise hardware.

SpinQuant makes extreme low-bit LLM inference practical: big memory and latency savings with near-full accuracy, using a small calibration step and without changing model APIs.

Cutting cold-start time from minutes to seconds reduces user-visible latency, lowers GPU idle costs, and increases successful request completion for large models, improving SLA and capacity efficiency.

If you run LLMs over tables in batches, reordering rows and fields can cut inference time and API bills materially by increasing prompt-cache reuse; it is a low-cost software change that often outperforms adding hardware.

Keyformer cuts memory traffic and latency for long-context generation without retraining, lowering inference cost and enabling higher throughput on existing GPU servers.

Mixing low-bit INT and FP per layer can keep model accuracy while cutting model size and quantization time; it fits current hardware that supports both INT and FP low-bit ops and reduces deployment cost.

FlexRound lowers precision without heavy retraining, letting you run large models with INT8/INT4 weights and small calibration sets while keeping near-original accuracy.