Abstract: In order to provide a simple and surreal burgeoning approach of interactive sand painting to ordinary users and artists, this paper presented an interactive algorithm to realistically simulate sand painting based on Kinect in real-time. The simulation algorithm consists of three subsystems: a preprocessing subsystem, a user interaction information acquisition subsystem, and a processing and rendering subsystem in real-time. The preprocessing subsystem includes the collection, analysis and statistics for real sand painting image and the definition for various brush styles. Both of these two operations need to be implemented only once. The acquisition subsystem as the starting point of our simulation algorithm could interactively recognize user action information, such as user gestures, by using Kinect. Based on the action information, the acquisition subsystem could know the function selection of users, and could recognize the painting style and the stroke paths in users’ painting. The processing and rendering subsystem adopted height-filed-based sand accumulation algorithm and sandpile collapse algorithm to simulate various painting styles, such as hand/fingertip sweeping, sand pouring and sand leakage. Furthermore, the height field of sand canvas would be converted back into a RGB image for the final effects rendering. The experimental results and the user experience feedbacks reveal that the proposed system and algorithm in this paper can generate sand painting artistic creations realistically, effectively, interactively in real-time.