FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
Paper
•
2506.20920
•
Published
•
75
Dataset
-
-
HuggingFaceFW/finewiki
Viewer • Updated • 61.6M • 9.12k • 271 -
nhagar/fineweb_urls
Viewer • Updated • 24.5B • 278 • 1 -
PleIAs/common_corpus
Viewer • Updated • 470M • 42k • 325 -
google-research-datasets/mbpp
Viewer • Updated • 1.4k • 2.01M • 201 -
allenai/c4
Viewer • Updated • 10.4B • 597k • 503 -
HuggingFaceFW/fineweb
Viewer • Updated • 52.5B • 188k • 2.6k