Two-Layer Motion Semantic Recognition by Fusing the Restricted Boltzmann Machine Based Generative Model and Discriminative Model

The semantic gap problem between the low-level features and high-level semantics often exists within the motion capture data. To tackle this problem, we refer to the deep learning theory and propose a two-layer motion recognition approach by fusing the Restricted Boltzmann Machine(RBM) based generative model and discriminative model, in which the generative layer is utilized for feature representation and the discriminative layer is selected for semantic discrimination. Within the proposed approach, we first utilize the autoregressive model to establish an one-way three-factored conditional RBM, whereby the spatiotemporal features of the captured motions can be well obtained. Then, these features are coupled with their corresponding labels and selected as the visible input of the RBM based discriminative model. Finally, by adding a voting space, the motion semantics can be efficiently recognized via this two layer fused model. The experimental results have shown that our proposed approach is able to recognize different kinds of motion poses, featuring robustness and expandability to the motion capture data. It is expected that the proposed approach would be well utilized for motion capture data reusing in a practical way.