Integration of gesture generation system using gesture library with DIY robot design kit

概要

Conversational agents are expected to improve the quality of communication by adding gestures to the speech, and are considered to be a promising tool. Recent data-driven methods are capable of attaching gestures to arbitrary speech, but the output is still not in line with human intuition. Therefore, we propose a gesture transformation system that utilizes gesture types as intermediate information, based on the theory of psycholinguistics. We employ the gesture-first principle to create gesture clusters based on gesture similarities among imagistic gestures, one type of gesture to represent image-like motions, which are considered to represent important concepts in conversations. Since this system explicitly takes into account the gesture types recognized by a deep neural network (DNN) and the semantics of the sentence to select gestures, it is expected to output gestures that are more in line with human intuition than existing end-to-end systems that do not place these intermediate states. We prepared a DIY robot kit consisting of cheap parts so that conversational agents at home become available to ordinary users, and implemented the proposed gesture generation system on this robot. In order to evaluate the effectiveness of the conversational agent, we evaluated user impression when using various media for conversation and confirmed the advantage of using our agent.

論文種別
発表文献
Proc.~IEEE/SICE International Symposium on System Integration (SII)
中島悠太
中島悠太
准教授

コンピュータビジョン・パターン認識などの研究。ディープニューラルネットワークなどを用いた画像・映像の認識・理解を主に、自然言語処理を援用した応用研究などに従事。