Abstract: Intelligent video surveillance has been developed rapidly and broadly in daily public security and various civil applications.To establish an effective and standard performance evaluation,an appropriate metric is very important.This survey summarizes the state-of-the-art of performance evaluation for intelligent video algorithms,especially focuses on the performance evaluation of motion detection and object tracking.It also highlights the future direction for performance evaluation of intelligent video algorithms.