Abstract: Based on statistics theory, a generic method for video multi-granularity semantic analysis is proposed in this paper, where multi-level semantics analysis and multi-modal information fusion are unified to represent temporal content, a key-frame selection strategy with temporal semantic context restriction and an attention selection model are presented firstly.After recognizing basic visual semantics, a framework for multi-level visual semantics analysis is introduced for visual semantics extraction.Then, Hidden Markov model and Bayesian decision are applied to audio semantic understanding.Finally, a bionic multimodal fusion scheme with two level structures is used for video semantic information fusion.Experimental results demonstrate the effectiveness of the proposed method to fuse multimodal features, as well as to extract video semantics with different granularity.