Most AI coding assistants scatter their conversation history across JSONL blobs, VSCode-like SQLite stores, and tool-specific folders — making comprehensive dataset collection tedious or incomplete. This toolkit flips that problem: it systematically finds installations, parses each storage format, and normalizes full conversations (messages, code context, diffs, tool runs) into a single JSONL output suitable for training or auditing.
What Sets It Apart
- Broad, practical coverage: scripts target Cursor, Claude Code, Codex, Windsurf, Trae, Gemini CLI, OpenCode and similar tools, so you can extract multi-tool histories without writing custom parsers — which saves weeks of ad hoc scraping work.
- Preservation of developer context: captures file paths, selected snippets, line ranges, suggested diffs and tool execution outputs, not just plain text prompts — meaning datasets retain actionable code-level context for fine-tuning or evaluation.
- Simple, file-based output: writes timestamped JSONL lines where each conversation bundles messages and metadata, making downstream filtering, deduplication, and ingestion into ML pipelines straightforward.
- Privacy-first guidance: the README and tooling emphasize secret scanning and safe storage practices so teams avoid accidentally training on API keys or proprietary code.
Who It's For and Trade-offs
Great fit if you need reproducible, large-scale extraction of local assistant usage for model training, dataset curation, or internal auditing. It’s designed for engineers and data scientists comfortable handling raw dumps (SQLite/JSONL) and applying secret-sanitization before downstream use. Look elsewhere if you need a hosted telemetry product, real-time ingestion, or turnkey anonymization workflows — this repo focuses on discovery and extraction, not on managed hosting or automated PII redaction.
Where It Fits
Use it as a data-collection layer in an ML data pipeline: run local extractions, sanitize and filter the JSONL outputs, then feed into deduplication, annotation, or fine-tuning steps. The project is a pragmatic bridge between scattered local assistant data and ML-ready datasets.
