Most design datasets capture only final artifacts, leaving out the sequence of decisions that produce them. This dataset records full expert sessions and converts them into long, intent‑segmented action trajectories so models can learn the workflow and creative decisions behind UI and layout design rather than just the end result.
What Sets It Apart
- Trajectory-focused supervision: 3,469 trajectories (2,883 train / 586 eval) derived from 200+ hours of expert Figma sessions, each a sequence of action-frame pairs in the Playwright‑MCP action space with inserted observe probes for input‑free transitions. This lets agents learn stepwise interaction patterns, not only final screenshots.
- Skill and phase structure: 126 long‑horizon tasks across 8 workflow categories and a 10‑skill expert taxonomy; trajectories carry closed 12‑label phase annotations to preserve high‑level workflow stages during training.
- Careful preprocessing for agent compatibility: ~95% idle mouse movements filtered, two‑pass frame extraction with settle detection, effect filtering by changed‑pixel fraction, and phase segmentation via a VLM consensus method; dataset compaction reported at 179× versus raw OS events.
- Practical for VLM/GU I agent work: formatted for supervised fine‑tuning and evaluation of vision‑language and GUI agents (e.g., long‑horizon, prototype wiring, sketch→Figma, theming, accessibility remediation).
Who It's For and Tradeoffs
Great fit if you want to fine‑tune or evaluate vision‑language models or embodied GUI agents on realistic, intent‑segmented human design workflows and long‑horizon interaction traces. It offers structured phase labels and action-level trajectories useful for learning agentic navigation and design best practices. Look elsewhere if you need pre‑annotated chain‑of‑thought reasoning traces (trajectories do not include explicit reasoning), if you require artifact‑only datasets (this focuses on workflows), or if you need fully noise‑free action logs (some noisy actions can leak through preprocessing and open‑ended tasks reflect individual SME preferences).
