高级检索

体型感知的音乐驱动舞蹈动作风格化生成

Shape-Aware Stylized Dance Motion Generation Driven by Music

  • 摘要: 高质量舞蹈动作自动生成在电影和动画制作等产业需求广泛. 然而已有方法生成的动作往往难以适应不同虚拟角色的骨骼比例和几何外形, 且无法对舞蹈动作风格进行灵活控制. 为解决上述问题, 提出体型感知的音乐驱动舞蹈动作风格化生成方法. 首先基于体型感知的运动重定向方法建立包含不同骨骼比例和几何外形的虚拟角色舞蹈动作数据集DanceDB++; 此外, 提出体型感知的音乐驱动舞蹈动作生成扩散模型, 利用交叉注意力机制将音乐特征与虚拟角色体型参数融合作为扩散模型的控制条件, 并引入考虑骨骼比例的辅助损失项; 同时, 引入语义风格引导模块, 从文本或示例动作中提取风格表示并通过自适应实例归一化层将其注入扩散模型, 以实现对生成动作风格的控制. 在DanceDB++上的定量实验结果表明, 所提方法在足部物理接触得分等评估指标上的表现优于对比的舞蹈动作生成方法; 在定性实验中, 展示了一系列文本描述或示例动作控制舞蹈动作风格的生成结果, 表明所提方法在生成舞蹈动作时能够实现有效的风格化引导. 文中提出的数据集和基于计图框架的模型代码将会开源.

     

    Abstract: The demand for high-quality dance motion generation is widespread in the film and animation industries. However, existing methods fail to adapt to varying skeletal proportions and geometric shapes of different virtual characters, and lack the flexibility to control the dance style. To address these issues, we propose a method for shape-aware music-driven stylized dance motion generation. First, we propose DanceDB++, a dance motion dataset of virtual characters with diverse skeletal proportions and geometric shapes based on a motion retargeting method that considers the body shape of character. Furthermore, we propose a shape-aware, music-driven dance motion generation diffusion model, which leverages a cross-attention mechanism to fuse the music features and the body shape parameters as the condition for the diffusion model, and introduce auxiliary training objectives based on skeletal proportions. Additionally, we introduce a semantic style guidance module that extracts style representations from text or example motions and injects them into the diffusion model through adaptive instance normalization layers, enabling flexible control over the generated dance style. The quantitative experimental results on DanceDB++ indicate that the proposed method outperforms the compared methods in metrics like physical foot contact score. In qualitative experiments, a series of stylized dance motions controlled by text descriptions or example motions show that the proposed method can effectively achieve stylized guidance in dance motion generation. The dataset and code based on Jittor framework will be publicly released.

     

/

返回文章
返回