Controllable video generation needs benchmarks that expose both scene structure and visual style while remaining editable; CodeVideoBench fills that niche by shipping runnable Three.js scenes alongside proxy videos and reference images so researchers can evaluate how well generation methods follow code-level structure while adopting target appearance.
What Sets It Apart
- Executable scenes: each case includes an editable Three.js program that reproduces scene layout, object motion, and camera trajectories, making the dataset suitable for code-to-video pipelines and interactive debugging. This means you can change geometry or timing and re-render a new proxy to test method sensitivity.
- Paired assets for three-way evaluation: every case provides a proxy video (960×544, 124 frames at 24 FPS), a high-resolution reference frame (1536×864), and a prompt, enabling separate scoring of structure (spatiotemporal alignment), style (appearance fidelity), and quality (visual/temporal realism).
- Baseline controls and reproducibility: the repository includes baseline control videos (depth/tracking) and Programs/ directories with source scenes and shared dependencies, facilitating consistent comparisons across methods and local inspection via a browser-based viewer.
- Clear licensing split: dataset content (Cases/ and BaselineControls/) is provided under CC BY 4.0 while project-authored code and docs use MIT, simplifying reuse in research projects.
Who It's For and Tradeoffs
Great fit if you develop or evaluate text-to-video or code-guided video systems and need editable, scene-grounded test cases to probe structure vs. style tradeoffs. It is less useful if you only need large-scale natural video corpora (CodeVideoBench is small—22 cases—designed for targeted, controllable evaluation rather than pretraining). Expect to spend time adapting Three.js scenes if your pipeline uses different scene representations or renderers.
