Benchmark dataset for evaluating long-horizon coding agents and software-engineering tasks, containing English code and tabular metadata in Parquet format; small scale (<1K examples) for fast prototyping and evaluation.
Provides agentic instruction‑tuning trajectories for software‑engineering tasks, formatted for supervised fine‑tuning and agent training. Contains multi‑file edits, tests, docs and structured agent traces (≈5,115 records, 1.9 GiB). Intended for commercial use; licensed CC‑BY 4.0 with additional permissive licenses.
Provides per-decision training samples for RL-driven command-line LLM agents: each record pairs a task prompt plus terminal history with a teacher's next-action in Terminus-2 JSON. Around 31k verifier-passing samples from 630 ATCB tasks, formatted for NeMo Gym's terminus_judge and licensed CC-BY-4.0.
Provides a curated benchmark of 170 real-world, multilingual code-refactoring instances to evaluate AI coding agents on large-scale, behavior-preserving, cross-file refactors. Each task includes rewritten issue descriptions and manually reviewed test suites to avoid over- and under-constraining evaluations.
Provides a queryable dataset of 3,797,117 SKILL.md agent-skill files found on public GitHub, deduplicated by content hash and enriched with representative text, front matter, folder composition, repo metadata, and sampled commit history for research.
Turns natural-language PLC requirements into verified, runnable IEC 61131-3 Structured Text by driving a closed loop of generation, compilation, deployment, and behavioral verification on a live OpenPLC runtime. The verification-gated harness forces inputs, traces execution, repairs failures, and renders ladder diagrams plus process simulation to raise dynamic runtime pass rates.
Evaluates whether coding agents can modify real scientific software while preserving domain-specific scientific contracts. Contains 119 repository-level tasks across 98 GitHub projects and 20 scientific domains, measures reproducible edits in pinned Docker images, and analyzes recurring failure modes.