Sensors, Vol. 19, Pages 3579: An Efficient Three-Dimensional Convolutional Neural Network for Inferring Physical Interaction Force from Video

Sensors, Vol. 19, Pages 3579: An Efficient Three-Dimensional Convolutional Neural Network for Inferring Physical Interaction Force from Video Sensors doi: 10.3390/s19163579 Authors: Dongyi Kim Hyeon Cho Hochul Shin Soo-Chul Lim Wonjun Hwang Interaction forces are traditionally predicted by a contact type haptic sensor. In this paper, we propose a novel and practical method for inferring the interaction forces between two objects based only on video data—one of the non-contact type camera sensors—without the use of common haptic sensors. In detail, we could predict the interaction force by observing the texture changes of the target object by an external force. For this purpose, our hypothesis is that a three-dimensional (3D) convolutional neural network (CNN) can be made to predict the physical interaction forces from video images. In this paper, we proposed a bottleneck-based 3D depthwise separable CNN architecture where the video is disentangled into spatial and temporal information. By applying the basic depthwise convolution concept to each video frame, spatial information can be efficiently learned; for temporal information, the 3D pointwise convolution can be used to learn the linear combination among sequential frames. To validate and train the proposed model, we collected large quantities of datasets, which are video clips of the physical interactions between two objects under different conditions (illumination and angle var...
Source: Sensors - Category: Biotechnology Authors: Tags: Article Source Type: research