DENSO WAVE, Beckhoff Automation and ExaWizards have developed a double-armed multimodal AI robot which performs real-time control of a robot arm by deep learning
～ Exhibiting in “System Control Fair 2017” and “International Robot Exhibition 2017”
at Tokyo Big Sight ～
DENSO WAVE INCORPORATED (Chita-gun, Aichi, President & COO: Hiroyasu Nakagawa, hereafter “DENSO WAVE”), Beckhoff Automation K.K. (Yokohama City, Kanagawa, President & CEO: Toshimitsu Kawano, hereafter “Beckhoff Automation”), and ExaWizards Inc. (Bunkyo-ku, Tokyo, Representative Director & President: Ko Ishiyama, hereafter “ExaWizards”) have jointly developed a multimodal AI robot which performs real-time control of a double-armed robot arm equipped with multi-fingered hands to conduct multiple operations for handling objects of indeterminate form by the same robot arm, embodying a robot system in the near future.
When performing multiple operations for handling objects of indeterminate form using a double-armed robot arm equipped with multi-fingered hands in the conventional robot system, a vast amount of programming was required. However, the multimodal AI robot developed this time does not require programming by a user and it can make the robot learn in the same way as teaching from person to person, by using deep learning and VR (Virtual Reality) technologies.
By integrating VR technology with the robot operations, the VR teaching system using an omnidirectional camera to leverage the visual and haptic senses can intuitively teach trajectories to the double-armed robot arm, and then the system makes the robot learn the recorded trajectories and the sensor information obtained from the robot arm and hands, enabling the robot to memorize the operations conducted by a human, without complex programming. In this development project, we have been supplied with the multi-fingered hands from exiii Inc., a venture company, and have been receiving advice from a famous AI robot researcher, Professor Tetsuya Ogata of the Faculty of Science and Engineering, Waseda University, making this a case of open innovation with collaboration between industry and academia. Representatives of the three companies made the following comments.
DENSO WAVE President Nakagawa: “I have long wanted to make industrial robots easy to use. By intuitively operating and teaching robots by using the VR system to enable the robots to infer the operations being taught through deep learning, robots will be able to respond robustly to the position and shape of the workpiece, unexpected changes in conditions and so on, thus significantly increasing the applications of industrial robots. We will work on expanding the use of robots not only in factory operations but also in the fields of pharmaceuticals, medicine, agriculture, logistics and other diverse areas.”
Beckhoff Automation President Kawano: “In the efforts to integrate robots, AI and VR as a system, the challenge was often the cost and time required to develop dedicated hardware. This time, the three companies promoting open systems gathered and combined respective versatile element technologies such as ORiN, TwinCAT, EtherCAT and deep learning, and have quickly established a multimodal AI robot at a realistic cost without any hardware development. This is a symbol of the democratization of cutting-edge technologies. Also, in this system, the system for real-time processing of nervous system processing by the CPU and for parallel cognitive processing by the GPU has been implemented in the same IPC. We will apply this concept as an intelligent controller not only to robots but also to the creation of intelligence for various production goods.”
ExaWizards President Ishiyama: “ExaWizards has been providing solutions for promoting the use of AI, aiming to solve social issues such as the super-aging society. We have accumulated results in education and care support systems and industrial and medical applications. This multi-modal AI robot that embodies our AI is a symbolic solution that enables various on-site issues to be solved directly. As the working population continues to shrink, we cannot avoid using robots to create a society through collaboration between people and AI. We will contribute to the fourth industrial revolution not just by providing AI to enable robots to reproduce the operations by people but also by providing AI to raise robot literacy among people through interactions with easy-to-handle and intelligent robots.”
The multimodal AI robot will be exhibited in the booth of DENSO WAVE at International Robot Exhibition 2017 (11/29–12/2) at Tokyo Big Sight and in the booth of Beckhoff Automation at SCF 2017 (11/29–12/1). In the exhibition, a double-armed robot which completed learning by deep learning technology folds up towels and serves salad on a plate. Also, a VR teaching system using VR technology can be experienced.
From 14:00 on Wednesday, November 29, representatives and development managers of the companies engaged in the development will gather in the booth of DENSO WAVE to provide a press briefing and a demonstration session. This is an open meeting and all visitors are welcome. Please feel free to visit the booth.
Overview of the exhibition
- Name: International Robot Exhibition 2017
- Period: 10:00–17:00, November 29 (Wednesday) – December 2 (Saturday), 2017
- Venue: DENSO WAVE booth (booth number: IR1-29), East 1 Hall, Tokyo Big Sight (Tokyo International Exhibition Center)
- Name: System Control Fair 2017
- Period: 10:00–17:00, November 29 (Wednesday) – December 1 (Friday), 2017
- Venue: Beckhoff Automation booth (booth number: 1-48), West 1 Hall, Tokyo Big Sight (Tokyo International Exhibition Center)
Press briefing
- Date and time: November 29 (Wednesday), starting at 14:00 (reception starts at 13:45)
- Venue: DENSO WAVE booth (booth number: IR1-29), East 1 Hall, Tokyo Big Sight
- Application for participation: Please fill in the necessary items on the separately attached press briefing invitation and send it to DENSO WAVE, Matsumoto / Ichikawa (FAX: 0569-49-5484) by November 27 (Monday).
Company profile
(1) DENSO WAVE INCORPORATED
Representative: President & COO Hiroyasu Nakagawa
Address: 1 Yoshiike Kusagi Agui-cho, Chita-gun, Aichi 470-2297
URL: http://www.denso-wave.com
Business: Development, manufacture and sale of automatic data capture equipment, industrial robots, programmable controllers and other apparatus and systems
(2) Beckhoff Automation K.K.
Representative: President & CEO Toshimitsu Kawano
Address: Nisseki Yokohama Building 18th Floor 1-1-8 Sakuragicho, Naka-ku, Yokohama-shi, Kanagawa-ken 231-0062
URL: http://www.beckhoff.co.jp
Business: Import, sale, system development and provision of automated measuring and control equipment solutions
(3) ExaWizards Inc.
Company name: ExaWizards Inc.
Address: 5F, Sumitomo Hamamatsucho Bldg., Hamamatsu-cho 1-18-16, Minato-ku, Tokyo
Founded: February 2016
Representative: Representative Director & President: Ko Ishiyama
Business: Development of AI-enabled services for industrial innovation and solving social problems
URL: https://exawizards.com/
(Note) The contents of this news release are as of the date of the press release, and are subject to change without notice.