Abstract: Video retargeting has received considerable attention in image processing research recently. Due to the limited memory and extensive computation, it is difficult to retarget a large-size video. Frame-byframe retargeting cannot solve this problem because of low quality of spatial-temporal coherence. In this paper, we present a new seam-carving-based video retargeting algorithm. First, video frames are read one by one and energy map per frame are calculated based on the gradient. Then energy map of current frame is adjusted by considering a few previous frames in a cache replacement method. Finally, seam carving based on linear interpolation is performed to resize current frame. We repeat this procedure until all the frames are resized. Experimental results showed that our algorithm not only preserves the important content, but also maintains both spatial and temporal coherence of the resized videos.