Long video streams reveal a mismatch between learned temporal state and the geometric evidence needed for stable reconstruction: some cues are fleeting while others (scale, global rigidity) persist over arbitrarily long horizons. The core insight behind this work is counterintuitive but simple — you do not need persistent learned long‑range memory to achieve stable, metric‑scale streaming reconstruction. By designing predictions whose targets are reference‑frame equivariant and by recovering global pose and geometry via sequential composition, a strictly local learned state can generalize stably to very long sequences while keeping per‑frame memory and compute bounded.
Key Findings
- Local‑only state with equivariant targets: the model caches KV features from only the previous 11 frames, predicts a point map in the current camera frame plus an adjacent relative pose, and composes poses to recover a global trajectory. So what: training and inference complexity remain independent of total sequence length, avoiding cache saturation and unbounded memory.
- Drift mitigation by light refinement and loss design: a compact motion‑visual rotation refiner and a composition‑aware pose loss reduce accumulated rotation and scale drift when composing many local relative poses. So what: the method reduces long‑horizon error compared to prior streaming models that rely on persistent memory or large context windows.
- Empirical long‑horizon stability: evaluated on long‑sequence benchmarks (e.g., Oxford Spires) the approach yields substantially lower absolute and relative pose errors versus prior streaming methods while remaining efficient to run on a single GPU. So what: practitioners can process kilometer-scale streams with constant per‑frame resources and limited model state.
Who it fits and trade-offs
Great fit if you need causal, bounded‑memory 3D reconstruction from long camera streams (e.g., driving sequences or extended walk‑throughs) and prefer a simple, composition‑based global recovery pipeline without loop‑closure dependencies. Look elsewhere if your application demands immediate globally consistent dense maps at revisit time (you may want methods with explicit loop closure or large persistent memories), or if you require per‑frame outputs tied to a fixed global reference without sequential composition.
Where it sits relative to alternatives
Compared with approaches that learn persistent compressed memories or multi‑scale trajectory tokens, this work deliberately avoids learned long‑range state and instead leans on geometric equivariance and composition. That simplifies state management and training while still delivering strong long‑horizon robustness in practice.