r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation

Why this matters

Large-model distillation data is often fragmented by teacher, domain, or format; this release assembles and exact-retokenizes high-quality assistant traces from three frontier teachers into a single, audit-ready SFT package so researchers can train or evaluate generalist LLMs without rebuilding complex pipelines.

What Sets It Apart

• Multi-teacher synthesis at scale: 57,937 canonical traces drawn from Qwen3.8-Max (48,283), GLM-5.2 (5,307) and Kimi K3 (4,347), with provenance and teacher identity preserved for auditability. This makes it straightforward to study teacher mixture effects or isolate single-teacher ablations.
• Rich, production-ready training views: 24 Parquet configs include sft_balanced (default rebalanced sampling), domain-specialists (math, code, reasoning, tools), glm47_native pre-tokenized inputs/labels, and sft_tools/rl_tool_prompts for agent and RL workflows. Tokenization was validated across GLM-4.7-Flash, Qwen3-0.6B and Llama-3.1-8B.
• Tool trajectories and loss masking designed for safe SFT: real tool-call trajectories are included (5,909 rows) and two deterministic loss-mask methods (prefix-differential, structural-redaction) ensure assistant tokens are the only trainable labels with zero mask failures.
• Balanced curriculum options: sft_balanced reweights under-covered capabilities (tools, long context) at training time, plus a recommended multi-stage curriculum (SFT → tool specialization → optional RL) to get practical agent behaviors.

Who it's for and tradeoffs

Great fit if you need a ready-to-use distillation corpus for supervised fine-tuning, ablation studies on teacher mixtures, or training agent-tool behaviors with explicit tool-call traces. The dataset is particularly useful for math/code/reasoning-focused LLM development and experiments requiring exact retokenization. Look elsewhere if you require broad multilingual coverage (multilingual content is thin at ~0.8%) or very large long-context coverage (GLM long-context rows are a small fraction); also note that the package is a synthetic research corpus aggregating teacher outputs and upstream source licenses vary by row, so downstream licensing checks are necessary.

Where it fits

Use this as a generalist SFT backbone (start with sft_balanced), then specialize with sft_tools/sft_agent for agent behaviors and rl_tool_prompts for reward-stage RL. The canonical and token_stats views simplify auditing, exact-token loss computation, and cross-tokenizer analyses.