AIAny
Icon for item

GPT-5.6 Sol Coding & Debugging Traces

Provides live Codex-CLI agent run traces from GPT-5.6 Sol capturing coding, debugging, security reviews, and harness/seed workflows in cumulative next-action prefixes — suitable for supervised fine-tuning and analysis of tool-using coding agents.

Introduction

Why this matters

Long-horizon, tool-using agent behavior is difficult to supervise from reconstructed transcripts. This dataset preserves exact Codex CLI sessions from GPT-5.6 Sol so every tool call, file edit, test run, and judgement remains in sequence — enabling supervised fine-tuning and analysis that trains agents to act, verify, and self-correct in real development workflows.

What Sets It Apart
  • Live agent provenance: each row is a cumulative prefix from an actual codex exec session (no backfilled assistant turns), so models see the exact orchestration pattern of tool calls, outputs, and edits.
  • Production-style verification: accepted traces are independently replayed and acceptance checks run twice; security sessions include hidden-reference grading and blind repo audits against held-out findings.
  • Task diversity and scale: the snapshot provides 17,939 next-step rows from 1,474 accepted trajectories across build, debug, seed-authoring, model-judge, harness construction, and security review modes in many programming languages.
  • Training-ready format: delivered as a single traces.jsonl where each line is one supervised next-assistant-action prefix; tooling and schema details (e.g., tools as JSON-encoded string) are documented for direct ingestion.
Who it's for and trade-offs

Great fit if you want supervised fine-tuning or evaluation data for tool-using coding agents, secure-code-review models, or research on build-test-fix loops. It is especially useful when you need exact execution traces, acceptance-suite evidence, and model-judged verdicts rather than synthetic dialogues.

Look elsewhere if you need a representative sample of real-world failures (the export is success-filtered), broad internet-scale repo diversity, or raw unfiltered agent traffic. Consumers must also respect the CC BY 4.0 license and review generated code and third-party licenses before deployment.

Practical notes
  • Format: single traces.jsonl with one cumulative next-action example per line and a documented schema for messages, tools, assistant_step, splits, and provenance.
  • Languages & toolchains: multi-language coverage (Python, JS/TS, Go, Rust, Java, C/C++, PowerShell, assembly, etc.) and real compilers/test suites used in tasks.
  • Intended uses: supervised fine-tuning, evaluation of verification-driven completion, training secure-review agents, and studying long-horizon repository work.

Overall, the dataset is a focused, provenance-rich corpus for anyone building or evaluating LLM-based coding agents that must interact with real toolchains and verify results.

More Items

Hugging Face

Provides self-contained synthetic environments for training data-analysis agents. This slice contains 1,538 tasks, each graded with a rubric designed to evaluate analytical findings.

Hugging Face

Evaluates data agents on 100 tasks that require evidence-backed conclusions from raw files spanning multiple business systems. Each task comes from a synthetic world and tests either causal explanations or downstream impact analysis against a task-specific rubric.

Hugging Face

Diagnoses physical plausibility in generated videos by scoring adherence to 13 observable laws. It pairs benchmark prompts and model outputs with quality-controlled human ratings for fine-grained evaluation.