Survey of 84 recent papers mapping models, datasets, benchmarks and gaps for Indic languages

June 13, 20248 min

Overview

Decision SnapshotReady For Pilot

The survey compiles many practical, open datasets and compact models, making real deployments feasible, but coverage and standardization remain uneven across Indic languages.

Citations2

Evidence Strength0.78

Confidence0.80

Risk Signals8

Trust Signals

Findings with numeric evidence: 6/6

Findings with evidence refs: 6/6

Results with explicit delta: 1/4

Reproducibility

Status: Partial assets available

Open source: Partial

At A Glance

Cost impact: 65%

Production readiness: 60%

Novelty: 45%

Authors

Sankalp KJ, Vinija Jain, Sreyoshi Bhaduri, Tamoghna Roy, Aman Chadha

Links

Abstract / PDF / Data

Why It Matters For Business

Indic languages cover ~1.5–2 billion speakers; focused datasets and compact models enable local-language products with much lower compute and cost than retraining huge universal models.

Who Should Care

Summary TLDR

This is a focused literature review of 84 recent papers on large language model (LLM) work for Indic languages. It catalogs models, corpora, benchmarks, tools, and methods; highlights uptake of small efficient models, large parallel corpora, and many new benchmarks; and lists recurring gaps: data scarcity, script and dialect diversity, code-mixing, and inconsistent evaluation. The paper is a practical map for building or evaluating Indic NLP systems and for prioritizing dataset and benchmark work.

Problem Statement

Indic languages cover huge linguistic diversity but are under-served by LLM research. Progress is patchy due to limited high-quality data, many scripts and dialects, frequent code-mixing, and few standardized benchmarks. This review aims to summarize what exists, surface gaps, and offer a taxonomy to guide researchers and practitioners.

Main Contribution

Systematic review and curated bibliography of 84 recent papers on Indic LLM research.

A taxonomy that groups work into five areas: LLMs, Corpora, Benchmarks & Evaluation, Techniques, and Tools & Applications.

Key Findings

Number of papers reviewed

Numbers84 papers screened and summarized

Practical UseUse this paper as a compact index to 84 relevant Indic-language works instead of searching widely.

Evidence RefSection 2.3 and 3 (survey statistics)

Small efficient Indic models can match or beat larger general models on local-language generation

NumbersParamanu family: 13.3M367.5M params; claimed 2066× smaller yet outperform GPT-3.5 and others on Bangla/Hindi/Sanskrit

Practical UsePrioritize efficient, language-focused pretraining and tokenizer design to get strong generation quality without massive compute.

Evidence RefSection 3.1.1 (Paramanu summary)

Results

MetricValueBaselineDeltaSplit / DatasetEvidenceEvidence Ref
papers reviewed84survey corpusAuthors screened >1000 titles/abstracts and included 84 studiesSection 2.3
Paramanu model sizes13.29M367.5M params; context 1024Paramanu paper summaryParamanu family reported model sizes and context lengthSection 3.1.1

What To Try In 7 Days

Inventory which Indic languages and scripts your product needs and map to Samanantar, Vistaar, or language-specific corpora.

Test a small, language-specific model (e.g., Paramanu or instruction-fine-tuned Airavata/Tamil-LLaMA) before defaulting to large general LLMs.

Prototype PEFT adapters (LoRA/FLix) on an open model to measure gains with minimal compute.

Optimization Features

Token Efficiency
script-aware tokenizers mentioned (advanced Indic tokenizer in Paramanu)
Model Optimization
LoRA
Training Optimization
typology-guided pretraining and comparable corpora (to avoid multilinguality curse)

Reproducibility

Code AvailableNo
Data AvailableYes
Open Source StatusPartial
LicenseUnknown

Data URLs

Samanantar and many referenced corpora were released by original authors (see their papers)

Risks & Boundaries

Limitations

Survey depends on included 84 papers; rapidly evolving field may have new work beyond their cutoff.

Heterogeneous reporting in source papers prevents uniform quantitative comparison.

When Not To Use

When you need certified high-stakes accuracy in under-represented dialects without further validation.

When a specific Indic script or dialect was not covered by the referenced datasets or models.

Failure Modes

Tokenization inequality: some scripts use many more tokens, raising cost and shrinking context.

Cross-lingual contamination or benchmark leakage may inflate reported scores.

Core Entities

Models

Paramanu (Gyan AI Paramanu)BLOOMGlot500XGLMIndicBARTmT5XLM-RLLaMA / Llama2GPT-3.5 / GPT-4AiravataTamil-LLaMAIndicTrans2IndicWhisperBanglaT5

Metrics

WERBLEUspBLEUROUGEF1 / micro-F1ROUGE-LHuman evaluation ratings

Datasets

SamanantarLMSYS-Chat-1MBactrian-XCulturaXVistaarMASSIVEXL-SumJW300OdiEnCorp 2.0Samanantar (Indic–Indic pairs)

Benchmarks

MEGAVERSEMEGAMULTIQIndicXTREMEFLORES-101INDICXNLIIndicNLGBenLLM-EvalVistaar (ASR benchmarks)