KM2D: Method for Generating Dance Animation Driven by Dance Movement Primitives and Musical Semantics

The fluidity and expressiveness of dance creation are directly driven and influenced by musical rhythm and dance movement primitives. This paper proposes a dance generation method driven by dance movement primitive symbols and musical semantics. This method aims to generate natural and fluid dance movement sequences that are in harmony with the musical rhythm, based on given music segments and sequences of dance movement primitive symbols. First, a dance dataset was constructed. Unlike existing datasets that only include annotations for music and dance types, our dataset adds detailed annotations for dance movement primitives on top of the music and dance sequences. By providing detailed annotations for musical rhythm and dance movements, effective mapping between music, movement primitive symbols, and actual dance movements is achieved. Secondly, a dance sequence generation method based on a diffusion model is proposed. This model not only captures smooth transitions between movements but also ensures that the generated dance movement sequences are consistent with the musical rhythm. By specifying detailed movements and combinations of different movement primitives, the generated dance movement sequences align with the intended style and content, reducing inconsistencies and unpredictability caused by random generation. This approach also enriches and refines the dance performance. In the network model, an action primitive symbol feature extraction module was added to achieve more efficient feature representation and integration. To optimize the generation effects, we designed a loss function that includes action smoothness loss, music synchronization loss, action continuity loss, and physical plausibility loss. Experimental results indicate that the diffusion model-based method can generate diverse, natural, and fluid dance movement sequences that match the musical rhythm. Through both quantitative and qualitative evaluations, we validated the effectiveness and superiority of the proposed method in dance generation tasks. Additionally, this method can be extended to generate dance sequences of different styles and complexities.