Most egocentric datasets offer video or sparse pose labels; this release pairs long-form first-person video with continuous, multi-site IMU orientation streams, making it a rare resource for models that must reason about limb kinematics from a wearable viewpoint.
What Sets It Apart
- Dense, long-form pairing: 13,451 recordings spanning 1,273.8 hours of head-mounted footage directly linked to 24 Hz orientation data from a seven-sensor harness (both hands, forearms, shoulders, chest). This makes multi-minute temporal context and procedural tasks available for learning long-horizon manipulation and action structure.
- Torso-relative poses: a chest sensor provides a stable reference frame so you can compute chest⁻¹ · limb to get torso-relative arm poses that are invariant to head yaw — useful for imitation learning, robot transfer, and viewpoint-robust action conditioning.
- Practical data engineering: video (MP4s + metadata.parquet) and IMU (long-form Parquet, ~780M rows) are split across two repos for loader-friendly shards and streaming access. Shard indices let you fetch per-recording IMU without scanning the whole set.
- Open reuse terms: published under CC-BY-4.0 so commercial and research reuse is straightforward with attribution.
Who It's For and Tradeoffs
Great fit if you need: embodied-AI pretraining or fine-tuning (vision-language-action, imitation learning), cross-modal research (video ↔ IMU prediction), worker- and tool-use modeling, or robotics imitation from first-person demonstrations. The dataset supports long, continuous task arcs (preparation → execution → result), which many short-clip corpora lack. Look elsewhere if you require: balanced class distributions, large contributor diversity, or guaranteed per-frame raw IMU. Folding laundry dominates the corpus (≈67.5% of hours), contributors are few (27 people) and unevenly weighted, and only ~21% of recordings include raw accel/gyro/mag (orientation quaternions are present throughout). Also note the raw video is unredacted; incidental private content may appear and must be handled by downstream users.
Where It Fits
Use this as a mid-to-large scale complement to image-only egocentric datasets or high-frequency head IMU corpora: it is particularly valuable when you need synchronized limb orientation signals tied to long procedural video. If your goal is general-purpose, balanced activity classification across many contributors, treat this as a specialized resource rather than a drop-in benchmark.
Practical tips
- Join on recording_id between tracker-pov (video + metadata.parquet) and tracker-pov-imu (IMU Parquet). Filter metadata first (e.g., qc_status, has_raw_motion, has_chest, n_slots).
- Prefer contributor-based splits to avoid train/test leakage across the same person/home/rig.
- If you need raw accel/gyro/mag, filter metadata.has_raw_motion; only a minority of recordings include them.
- Mind class imbalance: upsample tail activities, or use the corpus primarily for representation learning and transfer.
