Google has introduced Gemini Robotics ER 2, an embodied reasoning model for robotics that combines video understanding, task orchestration and multi-robot collaboration. The model enables robots to understand the physical world, interact with humans, plan multi-step tasks and adapt their actions based on real-time situations.
Gemini Robotics ER 2 works as a high-level reasoning layer for robots, while lower-level Vision-Language-Action (VLA) models handle motor execution. The model can also call external tools, including Google Search and developer-defined functions, allowing robots to gather information and complete tasks using connected capabilities.
Compared with Gemini Robotics ER 1.6, the model adds improved video understanding, progress tracking and multi-robot collaboration. By analyzing continuous video feeds, robots can monitor their progress, recover from errors and determine when to move to the next step. The model can also reason about upcoming actions while a robot is performing the current task.
Gemini Robotics ER 2 capabilities
Gemini Robotics ER 2 acts as a physical agent that can orchestrate multiple steps, self-correct during tasks and adapt to unfamiliar situations. Developers can connect low-level control interfaces, including Vision-Language-Action (VLA) models and navigation APIs, as tools and stream multimodal video, audio and text inputs directly into the model.
The model supports:
- Natural language interaction with users
- Multi-step task planning and orchestration
- Real-time task progress tracking
- Self-correction and recovery from failed actions
- Tool calling through Google Search and developer-defined functions
- Coordination between multiple robots
Google evaluated Gemini Robotics ER 2 using simulated robots, real-world robot control and human tele-operation setups. The company said the model improves tool orchestration performance compared with Gemini Robotics ER 1.6 across real VLA, simulated VLA and human tele-operation control modes.
Gemini Robotics ER 2 integrates with the Gemini Live API through a bidirectional streaming endpoint designed for latency-sensitive robotics tasks. This allows the model to coordinate action models and robotics APIs while reducing delays between reasoning and execution.
Google demonstrated the model with Boston Dynamics’ Spot robot, where Gemini Robotics ER 2 controls Spot APIs, including navigation and manipulator movement, to fetch objects using natural language commands.
Google has also shared example code on GitHub to help developers configure and use the model for robotics applications.
Video understanding and task progress tracking
Gemini Robotics ER 2 improves video understanding and progress tracking, allowing robots to verify whether complex tasks, such as tightening a light bulb or tying a trash bag, are completed before moving to the next step.
The update focuses on two areas: progress classification and moment-finding.
Progress classification allows robots to track how far they have progressed toward completing a task. Gemini Robotics ER 2 evaluates each video frame and assigns it to one of five progress stages:
- 0–20%
- 20–40%
- 40–60%
- 60–80%
- 80–100%
By measuring task progress, the model provides real-time awareness and allows robots to adjust actions or retry failed steps without restarting the entire workflow.
Moment-finding enables the model to identify the exact video frame where an important event occurs, such as knowing when to stop pouring coffee into a cup. Gemini Robotics ER 2 can use this capability to switch between tasks, verify completion and suggest corrections when required.
The model achieves:
- 91.3% accuracy on moment-finding tasks
- 0.96-second mean absolute distance
- 4× faster execution compared with larger model categories
Multi-robot collaboration
Gemini Robotics ER 2 introduces multi-robot collaboration, allowing different robots to communicate through a shared semantic understanding and coordinate tasks. Different robots can contribute based on their capabilities, such as wheeled robots operating indoors and humanoid robots handling uneven terrain.
Google demonstrated this capability with Apptronik’s Apollo 2 humanoid robot and Franka F3 Duo, showing how multiple robotic systems can collaborate on shared tasks.
Improving spatial reasoning
Gemini Robotics ER 2 improves spatial reasoning across three areas:
- Success and failure detection: The model now works with raw video feeds instead of static images to identify failures during task execution, including spills, slips and misalignments.
- General instrument reading: The model expands support beyond circular dials and sight glasses to digital displays, linear scales, rulers and liquid thermometers. Google tested this capability across 10 different instrument types.
- Enhanced spatial Visual Question Answering (VQA): The model improves visual question answering through Gemini’s multimodal understanding capabilities.
Safety improvements
Gemini Robotics ER 2 improves capabilities related to Safety Instruction Following and Human Proximity benchmarks. These evaluations measure how models follow physical constraints during reasoning tasks and detect nearby humans.
Google said Gemini Robotics ER 2 can stop a humanoid robot when a person is nearby and resume operation only after the area is clear.
The company is also introducing a benchmark to evaluate robotics foundation models as safe VLA orchestrators. The benchmark measures their ability to:
- Enforce safety constraints
- Monitor the environment
- Assess physical feasibility
- Request human clarification when needed
Availability
Gemini Robotics ER 2 is available to developers through the Gemini API and Google AI Studio. It is also available in private preview on the Gemini Enterprise Agent Platform.