whats the leader board of LLMs in low resource languages? what's the best LLMs to use?
- User
whats the leader board of LLMs in low resource languages? what's the best LLMs to use?
Assistant
For low-resource language performance, there isn't a single definitive leaderboard, but here are the key benchmarks and top-performing models:
Main Evaluation Benchmarks:
- XTREME and XTREME-R: Cross-lingual benchmarks covering 40+ languages including many low-resource ones
- FLORES-200: Machine translation benchmark for 200 languages
- AmericasNLI: Natural language inference for indigenous languages of the Americas
- MasakhaNER: Named entity recognition for African languages
Top Performing Models (as of January 2025):
-
mT5 and mT0: Google's multilingual T5 variants show strong performance across many low-resource languages, especially mT0 which is instruction-tuned
-
XLM-RoBERTa: Still competitive for many cross-lingual tasks, particularly when fine-tuned on specific low-resource languages
-
BLOOM and mBLOOM: The 176B parameter BLOOM model and its variants have shown good multilingual capabilities including for underrepresented languages
-
Aya models: Cohere's Aya-101 covers 101 languages and was specifically designed with low-resource languages in mind
-
LLaMA-based multilingual models: Various community efforts have extended LLaMA models to support more languages
Practical Recommendations:
For most low-resource language tasks, I'd suggest starting with mT0 or Aya models as they're specifically optimized for multilingual instruction-following. The choice depends on your specific language - some models have better coverage for certain language families (e.g., African languages vs. Indigenous American languages).
You can check current rankings on the HuggingFace Open LLM Leaderboard's multilingual section, though coverage of truly low-resource languages remains limited in standardized evaluations.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.