What are the best AI datasets in 2026?
Answer in brief: The best AI datasets in 2026 depend on the task. For LLM pretraining, start with FineWeb (18.5T+ English tokens, ODC-By) or Dolma (3T tokens, ODC-BY). For instruction tuning, use Tulu 3 or SmolTalk. For code, use The Stack v2 or OpenCodeInstruct (CC BY 4.0). For vision and video, use LAION's Re-LAION-5B and LAION-BVD, plus COCO and Open Images (annotations CC BY 4.0). For speech, use LibriSpeech, Common Voice and People's Speech. For robotics, use Open X-Embodiment, DROID, AgiBot World and the LeRobot datasets. Most of these are hosted on the Hugging Face Hub, which is the best single place to get them. Licenses range from fully open to research-only, so read each dataset card before you train.
🏆 Top pick: Hugging Face Hub (Datasets)
Why it wins: The official dataset cards for FineWeb, Dolma, Tulu 3, SmolTalk, OpenCodeInstruct, The Stack v2, People's Speech and AgiBot World are all on the Hub, and Hugging Face says it also holds almost 300 TB of robot data in LeRobot format. Every dataset is a versioned Git repository with a dataset card, and you can filter by task, language and license. Downloading public datasets is free.
Last updated: 2026-11-09 · Independent research based on official dataset cards, project pages and provider documentation. We have not tested any of these sources hands-on.
The best single place to find AI datasets is the Hugging Face Hub. Use Kaggle for easy-to-use community datasets with free notebooks. For training language models, start with Common Crawl and the filtered corpora built from it, such as FineWeb and Dolma. For image, video and text data, look at LAION. The right choice depends on your task and on each dataset's license, so check both before you train on anything.
The best AI datasets by task
LLM pretraining datasets
| Dataset | What you get | Where to get it | License |
|---|---|---|---|
| FineWeb | 18.5T+ tokens of cleaned, deduplicated English web text from Common Crawl, with 10B, 100B and 350B-token samples | Hugging Face | ODC-By, plus Common Crawl's terms of use |
| FineWeb-Edu | English web text from the FineWeb team with a quality score on every document, plus the same three sample sizes | Hugging Face | ODC-By |
| FineWeb-2 | Multilingual web text, with one subset per language and script and train/test splits for each | Hugging Face | ODC-By |
| Dolma | 3T tokens of web, academic, code, book and encyclopedia text. The newest version is v1.7 (April 2024) | Hugging Face | ODC-BY, plus the terms of each source |
| Common Crawl | Petabytes of raw web crawls collected since 2008. The newest is CC-MAIN-2026-43 | commoncrawl.org / AWS | Common Crawl terms of use; rights in the underlying pages vary |
Instruction-tuning datasets
| Dataset | What you get | Where to get it | License |
|---|---|---|---|
| Tulu 3 SFT Mixture (Ai2) | 939,343 post-training examples; Hugging Face lists 639 models trained on it | Hugging Face | ODC-BY-1.0 for the collection; some subsets are CC BY-NC 4.0 |
| SmolTalk | About 1M synthetic SFT samples, used to build the SmolLM2-Instruct models | Hugging Face | New components are Apache 2.0; imported subsets keep their original licenses |
| OpenHermes 2.5 | About 1M instruction and chat samples | Hugging Face | No license declared on the official card |
Code datasets
| Dataset | What you get | Where to get it | License |
|---|---|---|---|
| The Stack v2 (BigCode) | 3B+ files in 600+ programming and markup languages (67.5TB full, 32.1TB deduplicated). Ships file IDs; the contents come from Software Heritage | Hugging Face (gated) | Each file keeps its original license; Software Heritage terms apply; bulk download needs an agreement |
| OpenCodeInstruct (NVIDIA) | 5M coding question-answer pairs with unit tests, a 6.4GB download | Hugging Face | CC BY 4.0; the card says it is ready for commercial use |
Vision and multimodal datasets
| Dataset | What you get | Where to get it | License |
|---|---|---|---|
| Re-LAION-5B | 5,526,641,167 text–image-link pairs, in research and research-safe versions | LAION / Hugging Face | Apache-2.0 according to LAION's release post; the linked images belong to third parties |
| LAION-BVD | 80M downloaded videos (10M hours), 55M captioned clips and 300M captioned frames | LAION / Hugging Face | Research use only, not commercial use |
| COCO | Common Objects in Context annotations and benchmarks | cocodataset.org | Annotations are CC BY 4.0; images fall under Flickr's terms |
| Open Images V7 (Google) | About 9M images with 16M bounding boxes on 600 classes, 2.8M instance masks, 61.4M image-level labels on 20,638 classes and 66.4M point labels | Official site | Annotations are CC BY 4.0 (Google LLC); images are listed as CC BY 2.0, but Google says to verify each image's license |
Speech datasets
| Dataset | What you get | Where to get it | License |
|---|---|---|---|
| LibriSpeech | About 1,000 hours of 16kHz read English speech from LibriVox audiobooks | OpenSLR (SLR12) | CC BY 4.0 |
| Mozilla Common Voice | Crowd-sourced speech, released one dataset per language (Scripted Speech 27.0) | Mozilla Data Collective | CC0-1.0 |
| People's Speech (MLCommons) | English speech recognition data: 1.5M rows in the clean training split and 5.48M in the dirty one | Hugging Face | A mix of CC-BY (2.0–4.0) and CC-BY-SA (3.0, 4.0) |
Robotics datasets
| Dataset | What you get | Where to get it | License |
|---|---|---|---|
| Open X-Embodiment | 1M+ real-robot trajectories across 22 robot types, pooled from 60 datasets at 34 labs, in RLDS format | GitHub | Materials are CC BY 4.0 and code is Apache 2.0; check each constituent dataset too |
| DROID | 76k demonstrations (350 hours) across 564 scenes and 86 tasks | Project site | Not stated on the official pages we read |
| AgiBot World Beta | 1M+ trajectories from 100 robots, 2,976.4 hours in total | Hugging Face (gated) | CC BY-NC-SA 4.0 |
| LeRobot datasets | Almost 300 TB of robot data in LeRobot format on the Hub, including FineART (40k+ two-arm episodes, 1,718 hours) | Hugging Face | Set by each dataset |
Need commercially licensed or custom-collected data instead of open datasets? See our sister guides: Best AI training data providers and Best robotics data providers.
The ranked list: where to get AI datasets
Hugging Face Hub (Datasets)
What it is: The largest hub built specifically for AI datasets. Each dataset is a Git repository with a dataset card, and many also have an interactive Dataset Viewer. You can filter by task, language and license.
Best for: Almost everyone building machine learning or AI models, especially for text, speech, vision and large training corpora.
Strengths: Versioned Git-backed repos, dataset cards, search filters, and public or private visibility for organizations. It also hosts major corpora such as FineWeb and Dolma.
Drawbacks: Most datasets are uploaded by the community, so quality and documentation vary. Some datasets are gated or have the viewer turned off.
Pricing: Browsing and downloading public datasets is free. Free accounts get 100GB of private storage. Paid plans add more storage: the public storage add-on costs $12/TB/month, and extra private storage is $18/TB/month at the base rate (per Hugging Face's storage documentation).
Kaggle Datasets
What it is: Google-owned Kaggle's public dataset catalog, connected to its hosted notebooks and competitions. Kaggle's homepage lists 755K public datasets (checked November 2026), up from the 500,000+ it reported in February 2025.
Best for: Learners, data scientists and applied projects that want data and free compute in one place.
Strengths: You can open a notebook straight from a dataset page, and notebooks offer optional free GPU/TPU. Downloads work through the official Kaggle CLI or the KaggleHub library. Most public datasets can be downloaded without authenticating.
Drawbacks: Quality varies between uploads. Each dataset has its own license chosen by the uploader, and compute quotas apply.
Pricing: Free to view, download and analyze public datasets.
Common Crawl
What it is: A nonprofit web archive with petabytes of raw web pages, metadata and extracted text, collected regularly since 2008. The newest crawl listed today is CC-MAIN-2026-43.
Best for: Teams building their own web-scale text corpora for language-model pretraining.
Strengths: Huge scale and new crawls throughout the year. Access through Amazon's hosting is free. It also offers URL and CDXJ indexes, web graphs, and a public opt-out ledger.
Drawbacks: It is raw material. Before training you need serious filtering, deduplication, safety processing, rights review and a lot of compute.
Pricing: Free to access. You pay for your own compute.
FineWeb (Hugging Face)
What it is: A filtered English web corpus built from Common Crawl and published on the Hugging Face Hub under the ODC-By license. It is split into one subset per crawl, plus 10B-, 100B- and 350B-token samples.
Best for: LLM pretraining and data-ablation research when you don't want to filter Common Crawl yourself.
Strengths: Filtering is already done, it has an open license, and the sample sizes are good for small experiments. It loads directly from the Hub.
Drawbacks: English only, and very large. It inherits whatever rights and privacy questions apply to the underlying web content.
Pricing: Free download.
Dolma (Allen Institute for AI)
What it is: An open pretraining corpus of 3 trillion tokens. It mixes web content, academic papers, code, books and encyclopedia material, and it was used to train AI2's OLMo models.
Best for: Researchers who want a fully documented, reproducible mixed-source pretraining corpus.
Strengths: ODC-BY license (since April 2024), a datasheet and paper, open-source curation tools, a stats table for each source, a 16.4GB sample of about 10 billion tokens, and a form for requesting removal of personal data.
Drawbacks: The newest version (v1.7) came out in April 2024, so most of its sources stop in 2023. The full set is 4.5TB compressed.
Pricing: Free download.
LAION (Re-LAION-5B and LAION-BVD)
What it is: A nonprofit that publishes open multimodal datasets. Re-LAION-5B (August 2024) has 5.5 billion text–image-link pairs with extra safety fixes. LAION-BVD (August 2026) is a video dataset with 1.3 billion video URLs, 80 million downloaded videos (10 million hours), 55 million captioned clips and 300 million captioned frames.
Best for: Pretraining vision-language and video-language models.
Strengths: Huge scale, both research and research-safe versions, and open tools for working with the data (img2dataset, CLIP Retrieval, CLIP Benchmark).
Drawbacks: The image data is mostly links, so you have to download and filter the content yourself. LAION has replaced older releases such as the original LAION-5B. LAION-BVD is for research only, not commercial use.
Pricing: Free.
Google Dataset Search
What it is: A search engine for datasets. It indexes datasets from thousands of repositories across the web using schema.org markup.
Best for: Finding scientific, government or niche datasets when you don't know where they're hosted.
Strengths: Searches across many repositories at once and helps you find datasets that can be cited.
Drawbacks: It only finds and links to datasets. It doesn't host them, and its coverage depends on publishers adding markup.
Pricing: Free.
Registry of Open Data on AWS
What it is: AWS's catalog of datasets you can use through AWS, from sources such as NASA, NOAA, NIH, EPA, AI2 and Data for Good at Meta. It includes usage examples and SageMaker Studio Lab tutorials.
Best for: Very large scientific, geospatial, genomics and web datasets that you want to process in the cloud.
Strengths: Cloud-scale data such as Common Crawl, Sentinel-2 and The Cancer Genome Atlas, with documented usage examples.
Drawbacks: Unless stated otherwise, AWS does not provide or maintain the datasets. Each one has its own third-party license.
Pricing: The registry is free to browse. Your compute costs depend on how you process the data.
UCI Machine Learning Repository
What it is: UC Irvine's long-running repository, with 690 datasets, including classics such as Iris, Wine Quality and Bank Marketing.
Best for: Teaching, classical ML benchmarks and tabular experiments.
Strengths: Well-known, frequently cited datasets with clear metadata, and new datasets are still being added.
Drawbacks: Datasets are small by modern AI standards, and it is not a source for training foundation models.
Pricing: Free.
OpenML
What it is: An open machine learning platform that stores all of its datasets in a uniform format, so they load the same way through its Python, R and Java clients.
Best for: Benchmarking and AutoML on tabular data.
Strengths: One loader for every dataset, data-quality statistics computed automatically, and integrations with mlr, Weka, MOA, PyTorch and TensorFlow.
Drawbacks: Mainly tabular data, and small next to modern AI corpora. This entry relies on OpenML's official documentation.
Pricing: Free.
Google BigQuery public datasets
What it is: Datasets hosted in BigQuery through the Google Cloud Public Dataset Program. Google pays for storage, and you query the data with SQL.
Best for: Analysts and ML engineers who want to query large structured datasets in SQL without downloading them.
Strengths: The first 1TB of queries per month is free. Access works through the console, the bq CLI, the API and client libraries.
Drawbacks: Requires a Google Cloud project and charges for queries above the free tier. The program has no SLA.
Pricing: Pay per query after the first free 1TB/month.
Azure Open Datasets
What it is: Microsoft's curated, preprocessed public datasets covering weather, census, holidays, public safety, location and transportation (for example, NYC taxi trips). They are built into Azure Machine Learning and Azure Databricks.
Best for: Adding features to predictive models, especially for teams already on Azure.
Strengths: The data is cleaned and enriched before it reaches you. You can load it from any Python environment without an Azure account.
Drawbacks: The catalog is small and focused on enrichment data, not model training corpora.
Pricing: Free datasets. Azure compute is billed separately.
Data.gov
What it is: The US government's open data catalog, listing 532,480 datasets as of today.
Best for: Public-sector, geospatial, economic and civic datasets.
Strengths: Huge official coverage, organized by agency, with metrics on usage.
Drawbacks: Formats and quality vary widely. Most of it isn't packaged ready for ML.
Pricing: Free.
Zenodo
What it is: A general research repository run by CERN since May 2013. It has no requirements on format, size or license, and it issues citable records through DataCite.
Best for: Finding the datasets published alongside academic papers, and publishing your own with a DOI.
Strengths: Citable records, embargoes and restricted access, and an API.
Drawbacks: It isn't designed around ML, so how well datasets are documented and how easy they are to find varies.
Pricing: Free.
Comparison table
| # | Source | Type | Best for | Cost to access | Licensing |
|---|---|---|---|---|---|
| 1 | Hugging Face Hub | AI dataset hub | Text, speech, vision, LLM corpora | Free (paid storage tiers) | Per dataset, filterable |
| 2 | Kaggle | Community hub + notebooks | Learning, applied projects | Free | Per dataset |
| 3 | Common Crawl | Raw web archive | Building pretraining corpora | Free (your compute) | Terms of use; underlying web rights vary |
| 4 | FineWeb | Filtered web corpus | English LLM pretraining | Free | ODC-By |
| 5 | Dolma | Mixed pretraining corpus | Reproducible LLM research | Free | ODC-BY |
| 6 | LAION | Multimodal datasets | Image/video-text pretraining | Free | Per release; links to third-party content |
| 7 | Google Dataset Search | Search engine | Finding niche/scientific data | Free | Set by each host |
| 8 | AWS Open Data | Cloud registry | Scientific/geospatial at scale | Free registry; compute billed | Third-party, varies |
| 9 | UCI | Academic repository | Classical ML benchmarks | Free | Per dataset |
| 10 | OpenML | Benchmark platform | Tabular benchmarking, AutoML | Free | Per dataset |
| 11 | BigQuery public datasets | Hosted SQL datasets | SQL analysis at scale | First 1TB/month of queries free | Per dataset |
| 12 | Azure Open Datasets | Curated cloud datasets | Feature enrichment | Free data; compute billed | Per dataset |
| 13 | Data.gov | Government catalog | Public-sector data | Free | Mostly open; check each |
| 14 | Zenodo | Research repository | Datasets behind papers | Free | Any; set by each depositor |
How to choose
- Start from the task, not the size. For LLM pretraining, look at Common Crawl, FineWeb or Dolma. For instruction tuning, look at Tulu 3 and SmolTalk. For code, look at The Stack v2 and OpenCodeInstruct. For vision-language work, look at LAION and Open Images. For speech, look at LibriSpeech and Common Voice. For robotics, look at Open X-Embodiment, DROID and the LeRobot datasets. For tabular benchmarks, use UCI or OpenML.
- Read the license and the source. A license on a collection may not cover every underlying work. Tulu 3 and SmolTalk mix licenses, Open Images licenses its annotations separately from its images, and some datasets (AgiBot World, LAION-BVD) are non-commercial or research-only. Check the dataset card, the upstream terms, and whether consent and personal data have been handled.
- Check how fresh it is. Look at version dates and data cutoffs. For example, Dolma's newest version came out in April 2024, while Common Crawl publishes new crawls throughout the year.
- Plan for compute and storage. Corpora measured in terabytes or petabytes mean real processing costs. Cloud-hosted options (AWS, BigQuery, Azure) let you process the data where it's stored.
- Prefer documented datasets. Dataset cards, datasheets, versioning and removal processes make data easier to trust and to defend.
- Need licensed or custom-labeled data instead? Open datasets may not cover proprietary or human-labeled needs. See Best AI training data providers, or Best robotics data providers for robot data.
FAQ
What is the best place to find AI datasets?
For most people, the Hugging Face Hub is the best starting point. It is built for AI datasets and lets you filter by task, language and license. Kaggle is a close second, especially if you want free notebooks. Use Google Dataset Search when you don't know where a dataset is hosted.
Are these AI datasets free to use commercially?
Not automatically. Access is usually free, but each dataset has its own license, and a license on a collection may not grant rights to every underlying work. Some datasets, such as AgiBot World (CC BY-NC-SA 4.0) and LAION-BVD (research only), rule out commercial use. Read the license and dataset card before any commercial use.
What datasets are used to train large language models?
Open LLM pretraining usually starts with web data from Common Crawl, then adds filtered corpora such as FineWeb and mixed corpora such as Dolma, which combines web, academic, code, book and encyclopedia sources.
What are the best instruction-tuning datasets?
Tulu 3 SFT Mixture (939,343 examples, ODC-BY-1.0 with some non-commercial subsets) and SmolTalk (about 1M synthetic samples, with its new components under Apache 2.0) are two of the most reused open options on Hugging Face.
What are the best code datasets for training?
The Stack v2 is the big code pretraining corpus, with 3B+ files in 600+ languages, gated, with per-file licenses. OpenCodeInstruct offers 5M coding question-answer pairs under CC BY 4.0 for fine-tuning.
Where can I find large image-text or video datasets?
LAION publishes Re-LAION-5B, with 5.5 billion text–image-link pairs, and LAION-BVD, a video dataset with 80 million downloaded videos and 55 million captioned clips. Both are aimed at multimodal pretraining research. For human-labeled detection and segmentation data, Open Images V7 has 16M bounding boxes on about 9M images, with annotations under CC BY 4.0.
What are the best open robotics datasets?
Open X-Embodiment (1M+ trajectories across 22 robot types), DROID (76k demonstrations), AgiBot World Beta (1M+ trajectories from 100 robots) and the LeRobot datasets on Hugging Face, which hold almost 300 TB of robot data.
Where can I find small, classic datasets for learning machine learning?
The UCI Machine Learning Repository (690 datasets, including Iris and Wine Quality) and OpenML are the standard sources. Kaggle is also good for beginner-friendly projects.
How do I download a dataset programmatically?
On Hugging Face, use the datasets or huggingface_hub Python libraries. On Kaggle, use the Kaggle CLI or KaggleHub; most public datasets need no authentication. On OpenML, use openml.datasets.get_dataset in Python. For BigQuery public datasets, use SQL through the console, the bq CLI or the API.
Do I need a cloud account to use cloud-hosted public datasets?
For BigQuery public datasets, you need a Google Cloud project, and the first 1TB of queries per month is free. Azure Open Datasets can be accessed from any Python environment without an Azure account. Common Crawl data on AWS is free to access.
What is the difference between a dataset hub and a dataset search engine?
A hub such as Hugging Face or Kaggle hosts the data itself. A search engine such as Google Dataset Search indexes datasets hosted elsewhere and sends you to the original publisher.
Related guides: Best AI training data providers (licensed or custom-labeled data) · Best robotics data providers (commercial robot data).
More: AI dataset access costs compared · Methodology · About · Get featured
Named on this page as a provider? Join the network for a linked profile, Visit buttons and featured placement.