Many robot plans fail not because a model cannot identify objects, but because it cannot track how the scene will change across a chain of actions. The paper’s central idea is to make imagined future video part of the planning loop—and, in a separate extension, reuse the model’s predictive representations for control.
Key Findings
- The 16B base model combines an 8B autoregressive reasoner with an 8B video generator. Alternating subtask reasoning and predicted video lets it plan across successive steps rather than treating each action as an isolated response.
- On the introduced Rynn-VP benchmark, it scores 9.75 points above Cosmos 3 Nano overall. This tests whether visual anticipation can support extended plans, not just scene description.
- An RL-trained agent learns whether to answer directly or request generated visual evidence; it improves the Rynn-GEU average by 6.0 points over direct reasoning. Prediction is therefore treated as an optional reasoning tool, not an automatic step.
- The world-action extension reaches 51.5% success on 20 selected composite RoboCasa365 tasks and 46.7% average success across three real-robot tasks. These results show a link to physical control, while remaining bounded evaluations rather than evidence of general-purpose home robotics.
How the Planning Loop Works
The reasoner proposes a next subtask, describes the expected visual change, and estimates its duration; the generator predicts the corresponding video segment. That prediction becomes evidence for the next planning decision. For action, the world-action extension reads features from the video-generation process to produce robot controls, rather than requiring the robot to execute a fully decoded video frame by frame.
Who It Suits—and the Trade-off
A useful read for researchers working on embodied reasoning, video world models, and long-horizon manipulation, especially those asking whether prediction can improve action selection. Look elsewhere for a general-purpose robot system or broad deployment evidence: the reported robot results cover selected simulated tasks and three real-robot tasks, and generated predictions can sometimes mislead downstream reasoning.