LLMs write biased recommendation letters: women as warm, men as leaders

Overview

Why It Matters For Business

Automatically generated recommendation letters can embed gendered tone and hallucinated details that harm applicants and expose organizations to unfair hiring decisions and reputational or legal risk.

Summary TLDR

This paper measures gender bias in recommendation letters produced by large language models. The authors build a testbed of lexical and style metrics (formality, positivity, agency), a hallucination-detection pipeline (Context-Sentence NLI), and a balanced biography dataset (WikiBias-Aug). Evaluating ChatGPT and Alpaca, they find male-targeted letters are consistently more agentic, more positive, and often more formal; models also hallucinate biased content that amplifies those gaps. Results are statistically strong for agency and positivity (e.g., ChatGPT agency t=10.47, p≈1e-25). Use LLM outputs for real letters only after careful audit and human edit.

Problem Statement

People use LLMs to draft recommendation letters, but model outputs may embed gendered language and hallucinated facts that disadvantage applicants. The paper asks: do LLMs produce different word choices and styles for male vs female candidates, and do hallucinations amplify those biases?

Main Contribution

Key Findings

Results

Who Should Care

What To Try In 7 Days

Reproducibility

Risks & Boundaries

Core Entities

Related Papers

• BIASSCOPE: an automated LLM-driven pipeline that finds evaluation biases and builds a tougher JudgeBench‑Pro
• Pairwise comparisons amplify stylistic distractions; absolute scoring is more robust
• Psychometric audit finds durable provider-level biases in LLMs that can compound across multi-model systems
• JudgeBiasBench: a 12-type benchmark and bias-aware training to reduce LLM-judge bias
• Alignment makes LLM evaluators overuse certain scores; prompt score range is a cheap fix