ExaWizards and Taisei Corporation collaborate on a remote haptic transmission system and AI

– Confirming the autonomous operations of a robot arm –

ExaWizards Inc. (Representative Director & President: Ko Ishiyama) and Taisei Corporation (President & CEO, Representative Director: Yoshiyuki Murata) have jointly conducted experiments on testing by robot arm operation by combining the remote haptic transmission system developed by Taisei Corporation (hereafter “remote operation system”) and the multimodal AI developed by ExaWizards^*1 (hereafter “the AI”) and have confirmed that the remote operation system and the AI work in collaboration to enable a robot arm to operate autonomously. Following this result, we will work on using the combination for robot automation in various production sites.

Liquid weighing using the remote haptic transmission system employing AI

 With the recent social issues such as labor shortages and long working hours, there is growing demand for automation and manpower-saving by using industrial robots in production facilities, etc. However, time and cost are needed for “teaching” to record and learn the work to enable a robot to operate autonomously, and so despite the large potential need for robots, it has been difficult to introduce them in food factories where diverse items are handled and labor-intensive work is required, and also in pharmaceutical manufacturing facilities where high-mix low-volume products are handled in the quality control process, etc. Especially, liquid weighing has been considered difficult as the viscosity varies and as it is necessary to predict the change in state from the end of pouring to weighing.

 

Accordingly, ExaWizards and Taisei collected remote operation data preliminarily conducted by humans for specific tasks by the remote operation system, made the AI learn from the original data by deep learning as an example, constructed a model to learn the operations during work, and demonstrated that the robot arm was able to autonomously reproduce the same operations as those by a human.

 

The experiment for testing is outlined below.

 

[Test procedure]

The test procedure was implemented as follows for the series of weighing processes from pouring a certain amount of liquid from a wide-mouth bottle into a beaker, draining the liquid, and returning the bottle to its original place, by using a human collaborative robot (on the operation side) and a robot arm (on the remote side) equipped with a haptic transmission device to accurately convey the proper amount of force for holding an object, etc.

 

1. Implement remote direct teaching to continuously record and accumulate various data on the operations of a robot arm in the processes described above^*2 as original data for learning by AI.
2. Construct a learning model of a series of tasks through batch learning of the original data as described above on operations in a different data format by using multimodal AI.
3. Examine whether the robot arm can judge the state and autonomously perform liquid weighing by using the learning model.

 

[Test results]

1. For a series of liquid weighing processes, the AI batch-learned various data on robot arm operations continuously obtained for approximately 100 patterns, and thus achieved autonomous operations based on the learning model which controls the operations during work.
2. As collaboration between the remote operation system and the AI enables the robot arm to operate autonomously, it is expected to significantly reduce the time and cost required for automating a series of work processes with a robot arm.

 

Through this study, we confirmed that autonomous operations while predicting the future state according to the operations are possible. ExaWizards and Taisei will work together and collect various observations and knowledge for the development of related technologies and social implementation, aiming to implement robot automation technologies using the remote operation system and AI in diverse production facilities and production sites.

*1 Multimodal AI

AI which enables operations based on self-judgement, by interlocking multiple data elements required for the same operations as human operations by using a mathematical model (neural network) which works like a human brain, integrating these elements, and recognizing them as operation data.

 

*2 Various data on the operations of a robot arm

The following data related to weighing work continuously obtained by manual and direct operations by a human of the arm of a human collaborative robot: robot arm axial angle, current load, robot hand opening, tactile sensor value, camera video, values measured by electronic scale, etc.

 

[Profile of Taisei Corporation] (https://www.taisei.co.jp)

Address:              Shinjuku Center Bldg. 1-25-1, Nishi-Shinjuku, Shinjuku-ku, Tokyo

Founded:             December 1917

Representative:    President & CEO, Representative Director: Yoshiyuki Murata

Business:             Design, construction and installation of domestic and overseas building works and civil engineering works, and trade and business in diverse areas such as the environment, engineering, nuclear energy, urban development and real estate.

 

[Profile of ExaWizards Inc.] (https://exawizards.com/)

Company name: ExaWizards Inc.

Address: 5F, Sumitomo Hamamatsucho Bldg., Hamamatsu-cho 1-18-16, Minato-ku, Tokyo

Founded: February 2016

Representative: Representative Director & President: Ko Ishiyama

Business: Development of AI-enabled services for industrial innovation and solving social problems

＜Contact＞ 
ExaWizards Inc. Public Relation Division
E-mail：publicrelations@exwzd.com