Abstract: A novel multi-modality concept correlation graph approach is proposed to promote the performance of concept-based video retrieval.Firstly,a net-style concept correlation graph is constructed using the original similarities among the concepts.Then,query is added into the graph based on a multi-modality mapping strategy between query and concept.Thirdly,ranking on manifolds algorithm is used to dynamically diffuse the concept correlations in this graph until a global stable state is achieved.Finally,the selected concepts are orthogonally fused to implement the video retrieval.In the experiments of automatic video retrieval,our approach has achieved great improvement from 14.6% to 86.2% over the state-of-art methods.Moreover,we also apply it to our interactive retrieval system and gains excellent performance and user experience.