FineWeb review: one of the best AI datasets for LLM pretraining
Answer in brief: FineWeb is the open LLM pretraining dataset to beat for English. It has more than 18.5T tokens (originally 15T) of cleaned, deduplicated web text from Common Crawl, under the ODC-By license, with ready-made 10B, 100B and 350B-token samples for experiments. Its sister sets cover more ground: FineWeb-Edu adds a quality score to every document, and FineWeb-2 extends the approach to many languages. All three are free on the Hugging Face Hub.
Best for: English LLM pretraining and data-ablation research without filtering Common Crawl yourself.
Independent research from the official FineWeb, FineWeb-Edu and FineWeb-2 dataset cards, checked 2026-11-09. Not hands-on testing.
What it does for pretraining
FineWeb was built with Hugging Face's datatrove library, and its card says the pipeline is optimized for LLM performance. The card documents each step: URL filtering, text extraction with Trafilatura, a fastText language filter, Gopher repetition and quality filters, C4 quality filters, FineWeb's own custom filters, MinHash deduplication and PII formatting. FineWeb started as a fully open replication of RefinedWeb. The data is organized as one subset per Common Crawl dump, from CC-MAIN-2013-20 to CC-MAIN-2025-26, so you can train on specific time ranges.
The FineWeb family, by task
- FineWeb: English general web pretraining, 18.5T+ tokens, ODC-By.
- FineWeb-Edu: English web text with score and int_score fields on every document, so you can choose a quality threshold. Same 10B/100B/350B samples, ODC-By.
- FineWeb-2: Multilingual pretraining, with one subset per language and script and separate train and test splits. ODC-By.
For code, the FineWeb card points to The Stack v2.
Who it's built for
Teams pretraining small to large English models, researchers running data ablations, and anyone who wants a well-documented web corpus that is known to work.
Where it shines
- Ready to train. The heavy filtering and deduplication are already done.
- Sized for experiments. Use the 10B, 100B and 350B-token samples before you commit compute to the full set.
- Transparent. The card links its filter code, benchmark tasks, ablation models and evaluation results.
- A PII removal form lets people ask for their personal data to be removed.
Honest drawbacks
- The main set is English only; use FineWeb-2 for other languages.
- It's huge, so plan for serious storage and bandwidth.
- The newest crawl subset is CC-MAIN-2025-26, while Common Crawl is already at CC-MAIN-2026-43.
- It inherits whatever rights and privacy questions apply to the underlying web pages.
Pricing and licenses
Free to download. Licensed under ODC-By 1.0; the card also points to Common Crawl's terms of use.
Best alternatives
- Dolma: mixes web text with papers, code, books and encyclopedia material.
- Common Crawl: the raw source, with fresher crawls.
- Hugging Face Hub: where to find FineWeb's sister datasets and many more.
FAQ
How big is FineWeb?
More than 18.5T tokens of English web data (originally 15T), with 10B, 100B and 350B-token samples.
What license does FineWeb use?
ODC-By. The card also refers to Common Crawl's terms of use.
Is there a multilingual FineWeb?
Yes. FineWeb-2 is published under ODC-By, with one subset per language and script.
Sources
- FineWeb dataset card
- FineWeb-Edu dataset card
- FineWeb-2 dataset card
- datatrove
- Common Crawl terms of use
← Back to the ranked list of the best AI datasets
Named on this page as a provider? Join the network for a linked profile, Visit buttons and featured placement.