基于检索增强与适配器引导的草图到三维形状生成方法
A Sketch-to-3D Shape Generation Method Based on Retrieval Enhancement and Adapter Guidance
-
摘要: 针对草图信息稀疏、语义表达不足导致的三维形状生成精度和细节缺失问题, 提出一种基于检索增强与适配器引导的草图到三维形状生成方法, 包括基于检索的草图特征增强和适配器引导的形状控制生成2个部分. 基于检索的草图特征增强中, 通过检索相似彩色图像增强草图语义信息, 有效地提升生成精度与细节; 适配器引导的形状控制生成中, 提出一种草图—形状适配器, 结合交叉注意力机制实现草图特征与扩散过程的深度融合, 显著地提升生成形状与草图整体结构的一致性. 在ShapeNet-Synthetic和ShapeNet-Sketch子集上的实验结果表明, 所提方法在体素交并比(IoU)和倒角距离(CD)这2项指标上均取得最优性能, 平均IoU达到67.18%, 较现有最优方法提升1.05个百分点; 平均CD为0.003 97, 较最优方法降低0.007 53; 验证了检索增强机制和草图—形状适配器在提升整体结构一致性与细节还原方面的有效性, 为草图驱动的三维建模任务提供了新的技术路径.Abstract: Aiming at the problems of missing accuracy and details in 3D shape generation caused by sparse sketch information and insufficient semantic expression, this paper proposes a sketch-to-3D shape generation method based on retrieval enhancement and adapter guidance, mainly including two parts: sketch feature enhancement based on retrieval and shape control generation guided by adapter. The former enhances the semantic information of the sketch by retrieving similar color images, effectively improving the generation accuracy and details. The latter proposes a sketch—shape adapter, combined with the cross-attention mechanism, to achieve the deep integration of sketch features and the diffusion process, significantly im-proving the consistency between the generated shape and the overall structure of the sketch. Experiments on the ShapeNet-Synthetic and ShapeNet-Sketch subsets show that the proposed method achieves the op-timal performance in both the voxel Intersection over Union (IoU) and the Chamfer Distance (CD) indica-tors: The average IoU reaches 67.18%, which is 1.05 percentage points higher than that of the existing op-timal method; The average CD was 0.003 97, which was 0.007 53 lower than that of the optimal method. The experimental results verifies the effectiveness of the retrieval enhancement mechanism and the sketch-shape adapter in improving the overall structural consistency and detail restoration, providing a new technical path for the sketch-driven 3D modeling task.