Gemini Robotics Integrate AI Into The Physical World!
At Google DeepMind, we’re advancing how artificial intelligence (AI) solves complex problems through multimodal reasoning—connecting text, images, audio, and video. However, until now, these abilities have mostly been confined to the digital world. For AI to truly be useful in the real world, it must demonstrate “embodied” reasoning—similar to how humans understand and react to their surroundings, and then act on them safely. Today, we’re unveiling two ground breaking models based on our Gemini 2.0 system: Gemini Robotics and Gemini Robotics-ER.
What is Gemini Robotics?
Gemini Robotics is an advanced model built on Gemini 2.0, designed to control robots in the physical world through a process we call vision-language-action (VLA). Essentially, Robotics allows robots to process visual information, understand language, and take physical actions to perform tasks. With this model, robots can do a wider range of real-world tasks, adapting to different environments and challenges more seamlessly than ever before.
We’re also partnering with Apptronik to create next-gen humanoid robots with Gemini 2.0, and working with trusted testers to fine-tune our Gemini Robotics-ER model.
The Three Pillars of Gemini Robotics
To create robots that are truly helpful, we focus on three key qualities: generality, interactivity, and dexterity. Let’s explore how Gemini Robotics excels in each of these areas.
1. Generality: Adapting to New Situations
For a robot to be useful, it must generalize across various tasks. This means it should be able to tackle new tasks and objects that it hasn’t been specifically trained on. Robotics does this exceptionally well. In fact, it outperforms other state-of-the-art models by over 2x on a comprehensive generalization benchmark. Whether it’s folding clothes, assembling furniture, or handling delicate objects, Gemini Robotics can adapt and get the job done.
2. Interactivity: Responding to the Environment
To be useful in the physical world, robots must react to their surroundings quickly and accurately. Built on 2.0, Gemini Robotics is highly interactive. It understands conversational language, which means you can simply tell it what to do, and it will respond. It continuously monitors its environment, adjusting its actions as needed—whether you’ve moved an object or given a new instruction.
This adaptability allows robots to collaborate with humans in various environments, from homes to workplaces, by responding to instructions in real-time.
3. Dexterity: Handling Tasks with Precision
Performing complex, delicate tasks like folding origami or packing a snack into a bag requires fine motor skills. Most robots struggle with this, but Gemini Robotics excels in dexterity. It can manipulate objects with remarkable precision, making it suitable for tasks that require human-like handling.
Introducing Gemini Robotics-ER: Spatial Understanding Meets AI
While Gemini Robotics focuses on action, Gemini Robotics-ER is designed to enhance a robot’s spatial understanding. This model significantly improves 3D detection, spatial reasoning, and planning capabilities, enabling robots to carry out more advanced tasks.
For example, if shown a coffee mug, Robotics-ER can decide how to grip it properly using two fingers, ensuring a safe and efficient pick-up. It can also generate plans for how to approach and manipulate the object, all with minimal human guidance. This model is ideal for roboticists looking to connect the model with their own low-level controllers, allowing for customized robotic applications.
A Future Built on Safety
Safety is a top priority at DeepMind. As robots become more integrated into everyday life, it’s critical that they behave in a way that keeps people safe. That’s why we’re introducing new safety measures for Gemini Robotics-ER, including built-in features to prevent accidents. The model can assess whether an action is safe in its current environment and adjust accordingly.
We’re also releasing the ASIMOV dataset, inspired by Isaac Asimov’s famous Three Laws of Robotics, to help researchers evaluate and improve the safety of robotic actions. This dataset will guide the development of robots that are not only capable but also safer for human interaction.
Collaboration and Future Developments
In addition to our partnership with Apptronik, we’re working with a select group of trusted testers, including companies like Boston Dynamics and Agility Robotics, to refine Gemini Robotics-ER. Together, we aim to push the boundaries of AI and robotics, ensuring these technologies are both innovative and responsible.
Our goal is to build robots that can help solve real-world problems, whether that’s assisting in the home, the workplace, or beyond. The journey toward more capable, safe, and effective robots is just beginning, and we’re excited to continue developing these AI models for the future.
Conclusion
With Robotics and Gemini Robotics-ER, we are taking AI-powered robotics to the next level. These advanced models, built on Gemini 2.0, bring us closer to creating robots that can adapt, interact, and act with dexterity in real-world environments. By focusing on safety and collaboration with industry leaders, we’re shaping a future where robots work alongside us, helping us accomplish tasks in new and exciting ways. Stay tuned as we continue to explore and expand the possibilities of embodied AI in robotics.
