Argmin AIArgmin AI
About usOptimize your agent
RLHFBasic RAGPrompt OptimizationAgentic RAGQuantizationSafety BenchmarksLLM-as-a-JudgeChain-of-ThoughtMulti-Agent SystemsFunction CallingEfficient InferenceHallucination Detection

Soft prompts + frozen large LLMs are parameter‑efficient and better for cross‑site and few‑shot clinical extraction.

Prompt EngineeringPrompt TuningBenchmark RobustnessMethod
October 10, 20238 min

Overview

Decision SnapshotReady For Pilot

Results are strong on two established clinical benchmarks and include cross‑site and few‑shot tests, but experiments are limited to encoder (BERT‑style) LLMs and GPU cluster settings.

Citations1

Evidence Strength0.80

Confidence0.85

Risk Signals9

Trust Signals

Findings with numeric evidence: 5/5

Findings with evidence refs: 5/5

Results with explicit delta: 4/4

Reproducibility

Status: Partial assets available

Open source: Partial

At A Glance

Cost impact: 65%

Production readiness: 70%

Novelty: 45%

Authors

Cheng Peng, Xi Yang, Kaleb E Smith, Zehao Yu, Aokun Chen, Jiang Bian, Yonghui Wu

Links

Abstract / PDF / Data

Why It Matters For Business

Tuning only soft prompts on a frozen billion‑scale clinical LLM cuts compute and deployment costs while keeping or improving cross‑site and few‑shot extraction accuracy.

Who Should Care

CTOProduct ManagerML EngineerData ScientistEngineering Lead

Summary TLDR

The authors build a soft prompt (learnable continuous prompt) MRC model and compare four training strategies (no prompt, hard prompt unfrozen, soft prompt unfrozen, soft prompt frozen) across seven encoder LLMs and two n2c2 clinical datasets. Soft prompting usually beats hard prompts and traditional fine‑tuning. Freezing the LLM and only tuning soft prompts is parameter‑efficient and improves transfer and few‑shot generalization, but this only holds when the base model is large (billions of parameters). Small frozen models lose several F1 points.

Problem Statement

Clinical entity and relation extraction still depends on costly model tuning and manual prompt design. The paper asks whether learnable soft prompts plus frozen or unfrozen LLMs can reduce tuning cost while keeping or improving extraction accuracy and cross‑site/few‑shot performance.

Main Contribution

Introduced a soft‑prompt based machine‑reading‑comprehension (MRC) architecture for clinical concept and relation extraction.

Systematic comparison of four strategies: no prompt (fine‑tune), hard prompt (unfrozen), soft prompt (unfrozen), soft prompt (frozen).

Key Findings

Soft prompting with an unfrozen GatorTron-3.9B gave best concept extraction on drug‑ADE.

Numbersstrict F1 = 0.9118

Practical UseFor in‑domain concept extraction, tune soft prompts and model weights on ~3–4B models to get top accuracy.

Evidence RefTable 2 (2018 n2c2, Soft prompt; Unfrozen LLM)

Frozen billion‑scale models reach near‑parity with unfrozen tuning for concept extraction.

NumbersF1 ≈ 0.9085–0.9093 for GatorTron‑3.9B/8.9B when frozen

Practical UseYou can freeze a ≥3.9B clinical LLM and only train soft prompts to save compute and still keep high F1.

Evidence RefTable 2 (2018 n2c2: Soft prompt; Frozen LLM)

Results

MetricValueBaselineDeltaSplit / DatasetEvidenceEvidence Ref
Concept extraction (drug-ADE)F1=0.9118 (GatorTron-3.9B, soft prompt unfrozen)No prompt unfrozen F1=0.8883+0.02352018 n2c2 drug-ADETable 2 reports strict F1 scores; soft prompt unfrozen best at 0.9118Table 2
Concept extraction (SDoH)F1=0.8610 (GatorTron-8.9B, soft prompt unfrozen)No prompt unfrozen F1=0.8388+0.02222022 n2c2 SDoH (MIMIC-test)Table 2 shows soft prompt unfrozen GatorTron-8.9B at 0.8610Table 2

What To Try In 7 Days

Run a 2‑arm test: soft prompt frozen vs standard fine‑tune on your best available clinical model.

If you have ≥3B params access, train only soft prompts and measure strict F1 and cross‑site drop.

Experiment with prompt lengths (try 32 and 64 tokens) — performance varies 1–2%.

Optimization Features

Token Efficiency
Soft prompts are vector tokens added to embeddings (no extra tokenization)
Infra Optimization
Works with 8x A100‑80G GPUs in experiments
Model Optimization
Use frozen LLM + soft prompts to avoid weight updatesPrompt length tuning affects performance (32–64 preferred)
System Optimization
Frozen model deployment enables one model for multiple tasks
Training Optimization
Update only 2.5–6% of parameters when prompt‑tuning (paper reference)Five‑fold cross‑validation for hyperparameter selection

Reproducibility

Code AvailableNo
Data AvailableYes
Open Source StatusPartial
LicenseUnknown

Data URLs

https://n2c2.dbmi.hms.harvard.edu/

Risks & Boundaries

Limitations

Only encoder (BERT‑style) LLMs were examined; decoder/generative models not tested.

Experiments use clinical GatorTron family and may not generalize to all LLMs.

When Not To Use

Do not freeze and only prompt‑tune if your base model is small (≤345M): performance drops several F1 points.

Avoid relying on frozen soft prompts when you can afford full fine‑tuning and need marginal top accuracy in‑domain.

Failure Modes

Frozen small models underperform compared to unfrozen tuning (3.8–5.9% F1 drop).

Unfrozen tuning can overfit to institution data and hurt cross‑site performance.

Core Entities

Models

BERTBERT-MIMICRoBERTaRoBERTa-MIMICGatorTron-345MGatorTron-3.9BGatorTron-8.9B

Metrics

strict micro‑averaged F1-score

Datasets

2018 n2c2 drug-ADE (MIMIC)2022 n2c2 SDoH (MIMIC, UW)

Benchmarks

n2c2 2018 medication-ADEn2c2 2022 SDoH

You May Also Want to Read

PromptCBLUE: convert CBLUE into a prompt-format, multi-task Chinese medical benchmark and report baselines

0.60
0.45
0.60
4

PromptCBLUE gives a practical, Chinese-language testbed for medical LLM products. It shows that inexpensive PEFT fine-tuning of open 13B models can beat few-shot API use, so companies can invest in targeted fine-tuning to improve medical features.

Key finding

Fine-tuned open-source 13B models outperform few-shot commercial APIs on PromptCBLUE.

Numbers: Baichuan-13B (LoRA fine-tuned) overall 0.71 vs GPT-4 few-shot 0.518

Make one LLM-based user encoder serve many business scenarios by anchoring user profiles with queries and tiny soft prompts

0.80
0.45
0.80
0

One universal encoding plus small scenario prompts reduces model sprawl, lowers serving cost via KV-cache reuse, and improves live business metrics, so businesses can swap many specialized models for a single adaptable pipeline.

Key finding

Prompt-tuned Q-Anchor yields state-of-the-art discriminative and ranking performance on 10 real Alipay tasks.

Numbers: Avg AUC 0.8225; Avg KS 0.5267 (Table 2, C.1)

Prompt-tuning GatorTronGPT-20B gives efficient, higher-scoring clinical dialogue summaries

0.60
0.50
0.70
1

Prompt tuning lets teams deploy clinical summarization with much lower compute and faster turnaround than full fine-tuning while often improving quality if you have a large domain LLM.

Key finding

GatorTronGPT-20B prompt-tuned outperformed fine-tuned T5-Large on automatic metrics.

Numbers: Rouge-1 0.3628 vs 0.3425; BERTScore 0.7309 vs 0.6765 (Table 4)

Systematic review and 11-class taxonomy of 45 prompt optimization methods, datasets, and model gaps

0.40
0.35
0.45
0

Optimizing prompts often improves model outputs without costly retraining; however, inconsistent evaluations hide how well methods generalize, so businesses should validate prompt methods on their own balanced data before production.

Key finding

There are 45 distinct prompt optimization strategies covered by this review.

Numbers: 45 methods (reviewed)

Steer a frozen language model toward more commonsense using a small auxiliary head and a reference-free scorer

0.60
0.60
0.70
0

BOOST improves generation commonsense without fine-tuning large models, so teams can upgrade deployed LMs cheaply by adding a small controller and a scorer.

Key finding

Reference-free O-Scorer correlates with human commonsense ratings and matches top reference-based metrics.

Numbers: O-Score (mean) T5: 0.284, GPT-3.5: 0.299, Gold: 0.365; BERTScore-all: 0.302