AIAny
Icon for item

OM1

Turns camera, audio, LIDAR and web inputs into robot motion, navigation and speech by routing them through pluggable LLMs and VLMs. Hardware-agnostic Go runtime configured via JSON5, with ROS2/Zenoh middleware for real robots and simulators.

Introduction

Most robotics stacks force you down to C++ and low-level control loops. OM1 flips that: it treats a robot like an agent runtime — sensors in, LLM reasoning in the middle, motor commands out — and you wire the whole thing in JSON5 instead of writing control code. The key idea is a hardware abstraction layer where the same agent definition drives a humanoid, a quadruped, an educational bot, or a pure simulator.

What Sets It Apart
  • Pluggable model layer spanning OpenAI, Anthropic, Gemini, DeepSeek, xAI, Meta and Ollama, plus vision-language models. So what: you can swap the reasoning backend without touching robot code.
  • A Go runtime that replaced an earlier Python implementation for lower latency and a smaller memory footprint. So what: the agent loop runs on the edge device itself rather than a tethered workstation.
  • Middleware bridges to ROS2, Zenoh and CycloneDDS, with Prometheus and Grafana observability. So what: it drops into existing robotics infrastructure and lets you watch decisions in real time.
  • Behavior lives in JSON5 config, not source. So what: changing what the robot does is editing a file, not recompiling.
Who It's For

Great fit if you are a roboticist or AI developer prototyping embodied agents across Unitree G1/Go2, TurtleBot 4, or Gazebo/Isaac Sim and want to iterate on behavior quickly. Look elsewhere if you need deterministic real-time control or production safety guarantees: it is still beta, LLM-in-the-loop reasoning adds latency and nondeterminism, and out-of-the-box hardware support covers only a handful of platforms.

Information

  • Websitegithub.com
  • AuthorsOpenMind
  • Published date2025/01/08

More Items

Turns language commands into executable robot policies that combine perception, control APIs, loops, and geometry libraries. Hierarchical code generation recursively fills in missing functions to support new behaviors without policy retraining.

Learns executable language-conditioned local navigation by pretraining on instruction-conditioned Internet egocentric videos, aligning learned visual dynamics with simulated action labels, then jointly fine-tuning for navigation. Achieves high success rates on short-horizon benchmarks (86.5% OVON, 94.9% HabitatGS-LN) with real-robot validation and released code/data.

GitHub
AI Agent2026

Shows what fills a DeepSeek Harness agent's context and how it changes per request. Browse prompts, tools and messages, trace compaction, and inspect token usage, estimated costs and parent/subagent relationships.