TRI Revolutionizes Robot Skills with Breakthrough Diffusion Policy AI

In a significant stride toward enhancing the capabilities of robots, the Toyota Research Institute (TRI) has announced a pioneering generative AI technique known as Diffusion Policy. This innovative approach empowers robots to swiftly acquire new, intricate skills, thereby expanding their utility in various applications. This development is reminiscent of the transformative impact that large language models (LLMs) have had on conversational AI.

Gill Pratt, the CEO of TRI and Chief Scientist for Toyota Motor Corporation, emphasized the institute’s commitment to amplifying human potential through robotics. He stated, “Our research in robotics is aimed at amplifying people rather than replacing them. This new teaching technique is both very efficient and produces very high-performing behaviors, enabling robots to much more effectively amplify people in many ways.”

Previously, teaching robots new behaviors was a laborious and often inefficient process, marred by slow progress and limited applicability. Roboticists typically had to invest significant time in writing complex code or rely on countless trial-and-error iterations to program desired behaviors. However, TRI’s groundbreaking approach has changed the game.

TRI has successfully taught robots over 60 intricate and dexterous skills using the Diffusion Policy approach. These skills encompass a wide range of activities, including pouring liquids, using tools, and manipulating deformable objects. Remarkably, these achievements were accomplished without the need to code new instructions; the only requirement was providing the robot with new data. Building on this remarkable success, TRI has set ambitious goals to teach hundreds of new skills by the end of the year and an impressive 1,000 skills by the close of 2024.

Today’s announcement underscores the versatility of robots in adapting to new scenarios and performing diverse behaviors. These capabilities extend beyond simple “pick and place” tasks, allowing TRI’s robots to interact with the world in rich and varied ways. This versatility is a significant step toward enabling robots to support humans in everyday situations and unpredictable, ever-changing environments.

Russ Tedrake, Vice President of Robotics Research at TRI and the Toyota Professor of Electrical Engineering and Computer Science, Aeronautics and Astronautics, and Mechanical Engineering at MIT, expressed his excitement about the progress. He noted, “The tasks that I’m watching these robots perform are simply amazing; even one year ago, I would not have predicted that we were close to this level of diverse dexterity.” Dr. Tedrake highlighted the exceptional aspect of the new approach: its ability to rapidly and reliably acquire new skills, even in tasks involving deformable objects, cloth, and liquids, which have historically posed significant challenges for robots.

Revolutionizing robot skill acquisition

TRI’s robot behavior model learns through a combination of haptic demonstrations from a human teacher and a language description of the desired goal. Subsequently, an AI-based diffusion policy is employed to facilitate the learning of the demonstrated skill. This process enables the autonomous deployment of new behaviors based on dozens of demonstrations. Notably, this approach consistently delivers repeatable and high-performing results, all achieved at an impressive speed.

The Diffusion Policy approach, a collaborative effort between TRI and Professor Song’s group at Columbia University, represents a powerful generative AI technique for behavioral learning. It streamlines and accelerates the process of teaching robots new behaviors through demonstrations.

TRI’s robot platform is custom-designed to excel in dexterous dual-arm manipulation tasks, with a special focus on enabling haptic feedback and tactile sensing. This specialized platform provides the foundation for TRI’s achievements in expanding robot capabilities.

Looking ahead, TRI is determined to continue pushing the boundaries of robotics. The institute has already demonstrated the acquisition of 60 dexterous skills and aims to teach hundreds more by the end of the year, with an impressive target of 1,000 skills by the conclusion of 2024. This commitment to rapid advancement underscores TRI’s dedication to amplifying human potential through robotics.

Safety at the core of TRI’s robotics endeavors

Safety remains a paramount concern in TRI’s robotics endeavors. The institute has integrated strong safeguards into its systems, powered by the Drake framework and its custom robot control stack. These measures ensure that TRI’s robots adhere to safety guarantees, such as avoiding collisions with themselves or their environment.

The breakthrough Diffusion Policy approach has been detailed in a publication at the 2023 Robotics Science and Systems conference, providing insights into the technical aspects of this pioneering technique. Additional technical information can be found on TRI’s Medium blog.

To learn more about this groundbreaking advancement in robotics, interested individuals are invited to join a LinkedIn Live Q&A session on October 4th from 1 pm to 1:30 pm ET / 10 am to 10:30 am PT. This session will offer an opportunity to hear directly from the TRI robotics research team. To participate, please sign up for the event on TRI’s LinkedIn page.

Toyota Research Institute (TRI) is at the forefront of research aimed at enhancing human capabilities, with a focus on improving safety and sustainability. Led by Dr. Gill Pratt, TRI’s dedicated team of researchers is pioneering advancements in energy and materials, human-centered artificial intelligence, human interactive driving, machine learning, and robotics. Established in 2015, TRI maintains offices in Los Altos, California, and Cambridge, Massachusetts, where it continues to drive innovation and progress in the field of robotics.