高级检索
张欣天, 谢文军, 李书杰, 刘晓平. 基于卷积神经网络的Leap Motion运动数据优化网络[J]. 计算机辅助设计与图形学学报, 2021, 33(3): 439-447. DOI: 10.3724/SP.J.1089.2021.18425
引用本文: 张欣天, 谢文军, 李书杰, 刘晓平. 基于卷积神经网络的Leap Motion运动数据优化网络[J]. 计算机辅助设计与图形学学报, 2021, 33(3): 439-447. DOI: 10.3724/SP.J.1089.2021.18425
Zhang Xintian, Xie Wenjun, Li Shujie, Liu Xiaoping. Convolutional Neural Networks Based Motion Data Optimization Networks for Leap Motion[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(3): 439-447. DOI: 10.3724/SP.J.1089.2021.18425
Citation: Zhang Xintian, Xie Wenjun, Li Shujie, Liu Xiaoping. Convolutional Neural Networks Based Motion Data Optimization Networks for Leap Motion[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(3): 439-447. DOI: 10.3724/SP.J.1089.2021.18425

基于卷积神经网络的Leap Motion运动数据优化网络

Convolutional Neural Networks Based Motion Data Optimization Networks for Leap Motion

  • 摘要: 为提高Leap Motion设备的采集精准度,解决自遮挡、采样频率不稳定等设备固有问题,首先,设计了使用Leap Motion和动作捕捉设备的手部多模态同步运动采集方案,采集了日常动作数据集;其次,提出了基于卷积神经网络(convolutional neural network,CNN)的Leap Motion手部运动数据优化方法,使用日常动作数据集训练Leap Motion数据到动作捕捉数据的映射网络;最后,提出手指平面约束,确保网络输出数据保持稳定的手部骨骼结构.通过15名志愿者采集了6类动作共967550帧的同步运动数据集,进行了手指平面约束有效性、动作一致性实验,并与双向循环自编码器(bidirectional recurrent autoencoder,BRA)、双向编解码器(encoder-bidirectional-decoder,EBD)方法进行了精度对比.结果表明,文中方法支持使用Leap Motion获取固定采样频率且近似动捕设备精度的手部运动数据,效果较BRA和EBD更加稳定平滑.将文中方法应用于康复游戏中,明显减少了交互动作识别的错误次数.

     

    Abstract: It is necessary to improve the capture accuracy and precision of Leap Motion equipment,and solve its inherent problems such as finger self-occlusion and instable sampling frequency,firstly,a multi-modal synchronous hand motion capture scheme is proposed based on Leap Motion and motion capture devices,and the dataset is captured correspondingly.Secondly,a convolutional-neural-network-based hand motion data optimization method for Leap Motion is presented.The proposed network is trained to learn the mapping from Leap Motion data domain to motion capture data domain with the synchronous dataset.Finally,a coplanar constraint for human fingers is proposed,which makes the outputs maintain stable hand skeleton structure.The 967550 frames of synchronous hand motion data are captured in 6 categories from 15 volunteers.Experiments are designed for validating of the finger coplanar constraint,testing the consistency of the optimized motion data,and the comparison with BRA(bidirectional recurrent autoencoder)and EBD(encoder-bidirectional-decoder)methods.Experiments indicate that proposed method supports for capturing hand motion data with fixed sample frequency and MoCap-like precision using a Leap Motion.Furthermore,the proposed method can obtain smoother and more stable results than BRA and EBD.The method is also applied in rehabilitation games,which significantly reduces the number of errors of hand interaction motion recognition.

     

/

返回文章
返回