Why this matters Qwen-Drive-1.0 explores whether a general-purpose multimodal VLM can be adapted into a single foundation model that directly supports 3D BEV perception, driving visual question answering, and long-horizon motion planning. The project emphasizes preserving the pretrained VLM unchanged and attaching lightweight, inspectable modules so driving capabilities are added without catastrophic forgetting of general vision-language skills.
Key Capabilities
- Unified multimodal backbone: reuses a natively multimodal Qwen3.5-4B VLM as the shared encoder so visual-language reasoning and instruction-following are preserved.
- Explicit 3D probe (BEV perception head): fuses VLM and encoder features to produce BEV map segmentation, semantic occupancy, and 3D detection as an inspectable intermediate representation rather than a monolithic, opaque detector. Reported 3D numbers include ~43.95 mAP on nuScenes and strong BEV mIoU on multiple benchmarks.
- Planning Expert with flow matching: treats trajectory generation as conditional sequence generation (50 waypoints, 5s horizon at 10 Hz) and trains with flow-matching and RL reward tuning. Achieves high pseudo-closed-loop PDMS (90.7 on NAVSIM) and improved rater-alignment RFS (7.91 on WOD-E2E after RL).
- Staged, multi-source training recipe: unifies trajectory annotations across datasets, rewrites and filters driving VQA responses, and mixes driving supervision with general vision-language data to reduce catastrophic forgetting while adding driving-specific competence.
- Released artifacts: VLM weights plus three task heads (perception, planner-sft imitation head, and planner-rl reward-optimized head) for inference and further research.
Who it's for and trade-offs
Great fit if you are a researcher or engineer who wants a single multimodal foundation model that can a) answer free-form driving questions, b) output explicit BEV-style perception maps, and c) produce sampled future ego trajectories conditioned on VLM representations. The design is valuable for exploring VLM-centered AV pipelines and for experiments that require inspectable 3D outputs tied to language reasoning.
Look elsewhere if you need a production-grade, sensor-fused autonomous driving stack (full lidar/radar fusion, hardened safety validation, real-time control loops). The BEV head is intentionally simple (a probe) rather than an optimized state-of-the-art detector, and closed-loop driving readiness still requires substantial systems engineering and validation beyond the released research artifacts.
