Google has recently introduced a groundbreaking AI model, RT-2, which can translate vision and language into robotic actions. This Transformer-based model, trained on text and images from the web, can directly output robotic actions, bringing us closer to a future of helpful robots.
The Challenges of Robot Learning
Robot learning has always been a daunting task, requiring robots to handle complex, abstract tasks in highly variable environments. Unlike chatbots, robots need “grounding” in the real world and their abilities. Their training isn’t just about learning everything there is to know about an object but understanding how to interact with it in the physical world. This has historically required training robots on billions of data points, which is time-consuming and costly.
A New Approach with RT-2
RT-2 is a vision-language-action (VLA) model that removes the complexity of robots running on complex stacks of systems. It enables a single model to perform complex reasoning and output robot actions. This model shows that with a small amount of robot training data, the system is able to transfer concepts embedded in its language and vision training data to direct robot actions.
For example, if you wanted a robot to throw away a piece of trash, you would have to explicitly train it to identify trash, pick it up, and throw it away. However, with RT-2’s ability to transfer knowledge from a large corpus of web data, it already has an idea of what trash is and can identify it without explicit training. It even has an idea of how to throw away the trash, even though it’s never been trained to take that action.
A Brighter Future for Robotics
RT-2’s ability to transfer information to actions shows promise for robots to more rapidly adapt to novel situations and learn like we do — transferring learned concepts to new situations. In testing RT-2 models in more than 6,000 robotic trials, the team found that RT-2 functioned as well as our previous model, RT-1, on tasks in its training data, or “seen” tasks. And it almost doubled its performance on novel, unseen scenarios to 62% from RT-1’s 32%.
While there is still a tremendous amount of work to be done to enable helpful robots in human-centered environments, RT-2 shows us an exciting future for robotics just within grasp.
Stay Informed with Robotics Blogs
To stay informed about the latest developments in robotics, consider following these top 6 robotics blogs for industry news, trends, and insights:
- Robohub
- Robotics.org
- RoboGlobal News
- Robotics Business Review
- The Robot Report
These blogs offer a variety of expert perspectives analyzing issues and trends in the robotics industry, providing everything from big headlines to niche opinion pieces.
In conclusion, Google’s RT-2 model is a significant step towards practical, helpful robots that can learn and adapt to new situations. By staying informed about the latest developments in robotics, we can better understand and anticipate the future of this rapidly evolving industry.
