AIAny
Icon for item

Video models are zero-shot learners and reasoners

Argues a single web-scale generative video model handles vision tasks zero-shot the way LLMs handle language. Probes Veo 3 on segmentation, edge detection, image editing, physical and affordance reasoning, and puzzles like maze solving and symmetry.

Introduction

GPT-3 made a quiet point that took the field years to absorb: scale a single next-token predictor on enough text and task-specific models start to feel like a detour. This paper asks whether video generation is now at the same inflection point — whether predicting the next frame, at web scale, quietly bundles together the perception and reasoning skills that vision research has spent a decade building one specialist model at a time.

Key Findings
  • A single frozen model (Veo 3) is shown solving tasks it was never trained for, just by conditioning on a prompt and an input image — no fine-tuning, no task heads. The work treats prompting a video model as the visual analogue of prompting an LLM.
  • The capability sweep is unusually broad: low-level perception (segmentation, edge detection), pixel-level editing, physical-property and affordance reasoning, and abstract visual puzzles like maze navigation and symmetry recognition all fall out of the same generative objective.
  • The framing matters more than any single benchmark: it reframes a video model not as a clip generator but as a candidate unified vision foundation model, the way GPT-style models became unified language models.
Why This Reframes the Debate

Most of computer vision is still organized around bespoke architectures per task. The provocation here is that emergent, zero-shot generality — the thing that made LLMs eat NLP's task zoo — may already be appearing in video. If the trajectory holds, the interesting question shifts from "which model for segmentation" to "which prompt."

Who Should Read It

Great fit if you work on vision foundation models, generative video, or are tracking where the next paradigm shift in perception comes from — the chains-of-frames-as-reasoning angle is the takeaway. Look elsewhere if you need production-ready accuracy today: this is an existence-and-direction argument about emergent ability, not a claim that a prompted video model beats tuned specialists on any given metric.

Information

  • Websitearxiv.org
  • OrganizationsGoogle DeepMind
  • AuthorsThaddäus Wiedemer, Yuxuan Li, Paul Vicol, Shixiang Shane Gu, Nick Matarese, Kevin Swersky, Been Kim, Priyank Jaini, Robert Geirhos
  • Published date2025/09/24

More Items

Personalizes subject-driven videos to preserve human identity and accurate human–object interactions by integrating multimodal references and MLLM-derived semantics. Introduces global multimodal guidance in self-attention and modality-reference embeddings to align MLLM features with VAE tokens, supporting both inter- and intra-subject inputs (e.g., OCR, multi-view).

Evaluates whether video models reason according to physical laws by treating generated videos as visible reasoning traces and using a three-stage Perception–Formulation–Deduction protocol. Includes Orchard (400 mechanics videos), chain-of-frames prompting on annotated first frames, and a hybrid MLLM-plus-objective scoring suite for stage-resolved diagnostics.

Predicts variable-cardinality sets of evidence intervals in videos to temporally ground queries using multimodal large language models. Combines caption-derived multi-span supervision, a temporal Wasserstein matching-free reward, and temporal IoU, yielding strong mIoU gains across multiple benchmarks.