AI Coding2023
A statically typed, parallel programming language that prevents AI-introduced bugs by making correctness claims (laws) require machine-checked proofs. Compiles to native code and targets CPU/GPU parallelism with a fast proof checker.
AI Dataset2023
Evaluates difficult scientific question answering with expert-written multiple-choice questions in biology, physics, and chemistry. Its validation tests how well specialists and cross-domain PhD researchers perform, even with open-web access.
AI Dataset2024
Hisami Suzuki, Satoru Katsumata +4LLM-jp (Center for Large Language Model Research and Development, National Institute of Informatics), National Institute of Informatics
Provides manually curated Japanese instruction pairs (questions and safe reference answers) for improving LLM output safety, covering broad harm categories and regionally sensitive cases. Includes English meta-tags and standard splits for benchmarking and fine-tuning.
AI Dataset2024
A fine-grained, domain-labeled web corpus for NLP and LLM research that decomposes a deduplicated web crawl into 67 domains via iterative URL/content labeling and coarse→fine recall; includes per-domain statistics, validation subsets, and domain-benchmark analysis tools.
AI Audio2025
Generates full songs from lyrics and style prompts by first writing an editable melody-and-chord score and then rendering vocals and accompaniment. Unifies symbolic planning and audio synthesis in one pipeline, enabling readable score inspection, zero‑shot covers, and agent-driven editing.
AI Infra2025
Measures generative AI inference performance with token-level metrics (TTFT, inter-token latency), latency, and throughput under realistic traffic patterns. Provides a multiprocess engine, real-time TUI dashboard, extensible plugins, and integrations for telemetry and result uploads, aimed at inference benchmarking and capacity planning.
AI Agent2025
Orchestrates code-running agents to hunt verifiable open problems: schedules runs, applies null tests and skeptic re-checks, and packages found objects with short checkers so anyone can reproduce or validate a claim.
AI Agent2025
Evaluates and optimizes AI agents and language models in containerized environments, supporting large-scale parallel benchmarks and RL rollouts. Integrates with third‑party providers for thousands of parallel environments and serves as the official harness for Terminal‑Bench.
AI Dataset2025
Provides a reproducible, deduplicated corpus of text extracted from PDFs for LLM pretraining—about 3 trillion tokens from ~475 million documents in 1733 language-script pairs. Includes OCR and text extraction pipelines, per-page language IDs, MinHash deduplication, and is released under ODC‑By 1.0.
AI Dataset2025
Open synthetic corpus for training small reasoning-focused language models — ~79.65M generated samples (≈75B tokens with Pleias tokenizer) amplified from ~58.7k Wikipedia/Wikibooks seeds; includes explicit synthetic reasoning traces, multilingual coverage, and parquet splits.
AI Dataset2026
Provides a physical reconstruction benchmark of OmniDocBench v1.5 by producing five real-world photographic variants (Scanning, Warping, Screen‑Photography, Illumination, Skew) for each of 1,355 pages, inheriting original ground-truth to enable controlled, scenario-wise evaluation of document parsing robustness.
MCP Server2026
Provides a local MCP server that returns precise, symbol-level code (functions, classes, imports) via tree-sitter parsing so AI agents send only the bytes they need—commonly cutting code-reading token usage 95%+ and enabling compact packed responses for further savings.