gpt-4.1-2025-04-14· OpenAI LMArena Human Preference 140K
enumerate all text-based image retrieval datasets
- User
enumerate all text-based image retrieval datasets
- Assistant
Certainly! Here’s a comprehensive list of text-based image retrieval datasets (also called image-text retrieval or cross-modal retrieval datasets). These datasets are commonly used for evaluating and training models that retrieve images based on textual queries (and vice versa).
General-Purpose Datasets
1. MS COCO (Microsoft Common Objects in Context)
- Description: 330K images, each with 5 captions.
- Use: Image-text retrieval, captioning, VQA.
- Link: MS COCO
2. Flickr30K
- Description: 31K images, each with 5 captions.
- Use: Image-text retrieval, captioning.
- Link: Flickr30K
3. Flickr8K
- Description: 8K images, each with 5 captions.
- Use: Smaller-scale retrieval, captioning.
- Link: Flickr8K
4. Conceptual Captions
- Description: 3.3M images with captions harvested from the web.
- Use: Large-scale pretraining, retrieval.
- Link: Conceptual Captions
5. SBU Captions
- Description: 1M images with captions from Flickr.
- Use: Large-scale pretraining, retrieval.
- Link: SBU Captions
6. Visual Genome
- Description: 108K images with region descriptions, Q&A, and relationships.
- Use: Dense captioning, retrieval, VQA.
- Link: Visual Genome
7. Google Conceptual Captions 12M (CC12M)
- Description: 12M image-caption pairs.
- Use: Large-scale pretraining, retrieval.
- Link: CC12M
8. YFCC100M (Yahoo Flickr Creative Commons 100M)
- Description: 100M images/videos with metadata and user tags.
- Use: Large-scale retrieval, weakly supervised learning.
- Link: YFCC100M
Specialized/Challenging Datasets
9. RSICD (Remote Sensing Image Caption Dataset)
- Description: 10,921 remote sensing images, each with 5 captions.
- Use: Remote sensing retrieval.
- Link: RSICD
10. Fashion-Gen
- Description: 293K fashion images with detailed descriptions.
- Use: Fashion retrieval, generation.
- Link: Fashion-Gen
11. CUHK-PEDES
- Description: 40,206 images of 13,003 persons, each with 2 sentences.
- Use: Person search by natural language.
- Link: CUHK-PEDES
12. Oxford Flowers-102 Captions
- Description: 8,189 images of 102 flower categories, each with 10 captions.
- Use: Fine-grained retrieval.
- Link: Oxford Flowers-102
13. CUB-200-2011 with Captions
- Description: 11,788 bird images, each with 10 captions.
- Use: Fine-grained retrieval.
- Link: CUB-200-2011
14. Recipe1M
- Description: 1M recipes with images and text.
- Use: Cross-modal retrieval (food domain).
- Link: Recipe1M
15. MIR Flickr
- Description: 25K images with user tags and descriptions.
- Use: Multi-modal retrieval.
- Link: MIR Flickr
Recent Large-Scale/Noisy Datasets
16. LAION-400M / LAION-5B
- Description: 400M/5B image-text pairs scraped from the web.
- Use: Large-scale pretraining, retrieval.
- Link: LAION
17. RedCaps
- Description: 12M image-caption pairs from Reddit.
- Use: Large-scale pretraining, retrieval.
- Link: RedCaps
Other Notable Datasets
18. VG-COCO (Visual Genome + COCO)
- Description: Combines region descriptions from Visual Genome with COCO images.
- Use: Dense retrieval.
19. TextCaps
- Description: 145K images with captions focusing on text in images.
- Use: Scene text retrieval.
- Link: TextCaps
20. MSR-VTT
- Description: 10K video clips with 200K captions.
- Use: Video-text retrieval.
- Link: MSR-VTT
Summary Table
Dataset # Images/Pairs Captions per Image Domain Link MS COCO 330K 5 General COCO Flickr30K 31K 5 General Flickr30K Flickr8K 8K 5 General Flickr8K Conceptual Captions 3.3M 1 Web Conceptual Captions SBU Captions 1M 1 Web SBU Captions Visual Genome 108K Dense regions General Visual Genome CC12M 12M 1 Web CC12M YFCC100M 100M Tags/metadata Web YFCC100M RSICD 10.9K 5 Remote Sensing RSICD Fashion-Gen 293K 1+ Fashion Fashion-Gen CUHK-PEDES 40K 2 Person CUHK-PEDES Oxford Flowers 8K 10 Flowers Oxford Flowers-102 CUB-200-2011 11.8K 10 Birds CUB-200-2011 Recipe1M 1M 1+ Food Recipe1M MIR Flickr 25K Tags/desc. General MIR Flickr LAION-400M/5B 400M/5B 1 Web LAION RedCaps 12M 1 Web RedCaps TextCaps 145K 1 Scene Text TextCaps MSR-VTT 10K videos 20 Video MSR-VTT
If you need a more specific list (e.g., only for English, only for fine-grained, etc.), let me know!
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.