Abstract: A novel approach is proposed for video summarization.First,robust shot boundary detection is conducted by evaluating the inter-frame distance based on image color histogram and by removing the false positive based on camera motion analysis;then,each shot is classified as stationary shot,shot with object motion and shot with camera motion by analyzing its motion indicator map;we finally adapt well initialized kernel K-means clustering with a proposed multi-instance distance metric to each class of shots.The video summarization is constructed by extracting the shot nearest to cluster center in each cluster and by integrating them according to their occurring time instance in the original video.Compared with the state-of-art algorithms,our proposed approach can detect shot boundaries with higher accuracy,classify the shots according to motion information,and build more informative video summarization by abstracting each class of shots respectively.