AIAny
Icon for item

FinePDFs

Provides a reproducible, deduplicated corpus of text extracted from PDFs for LLM pretraining—about 3 trillion tokens from ~475 million documents in 1733 language-script pairs. Includes OCR and text extraction pipelines, per-page language IDs, MinHash deduplication, and is released under ODC‑By 1.0.

Introduction

Oops! Something went wrong

[next-mdx-remote-client] error compiling MDX: Unexpected character `2` (U+0032) before name, expected a character that can start a name, such as a letter, `$`, or `_` More information: https://mdxjs.com/docs/troubleshooting-mdx

Information

  • Websitehuggingface.co
  • OrganizationsHugging Face
  • AuthorsHynek Kydliček, Guilherme Penedo, Leandro von Werra
  • Published date2025/09/05

Categories

More Items

Hugging Face

Multimodal real-robot dataset of 28,993 teleoperated bimanual manipulation demonstrations (~300 hours) with synchronized multi-view RGB/tactile videos, 65D robot state/action, 60D fingertip tactile signals, and temporally aligned language segments; intended for imitation, visuomotor and visual‑tactile research, CC BY‑NC 4.0 (non‑commercial).

Hugging Face

Benchmark dataset for bilingual (Chinese/English) visual document image editing with 3,506 examples covering 1,578 text edits and 1,928 table edits. Each example pairs input and target images with editing instructions, region masks, and structured annotations for OCR-level evaluation.

GitHub

Charting engine and interpretation dataset for Zi Wei Dou Shu (Purple Star Astrology): deterministic natal chart algorithm, Four Transformations, pattern rules, and large v3/v4 interpretation datasets. Code MIT and data CC BY 4.0; judgment library and AI backend are closed.