时空信息联合嵌入的端到端三维模型草图检索
End-to-End Sketch-3D Model Retrieval with Spatiotemporal Information Joint Embedding
-
摘要: 现有的基于草图的三维模型检索工作往往将数据视为静态输入,并使用卷积神经网络进行特征提取,忽略了数据的动态属性,造成了有益信息的部分丢失,进而影响了以此为基础的检索效果.为解决这一问题,提出一种基于时空信息联合嵌入的端到端三维模型草图检索算法.首先,将草图表征为动态绘制序列,体现其绘制过程中所包含的时序信息;将三维模型表征为多视图序列,体现视图间的位置关联;然后,构建包含静态空间特征提取和动态时序特征提取的端到端双流网络,结合三元中心度量学习建立跨域数据的联合时空特征嵌入,充分捕捉草图和三维模型所包含的静态、动态特征,缩小跨域数据的差异性,提高检索的准确率;最后,在标准公开数据集SHREC2013和SHREC2014上进行实验,与现有工作相比获得了较高的准确率,验证了所提算法的可行性及有效性.Abstract: The existing sketch-based 3D model retrieval methods often regard data as static input,and utilize convolutional neural network to extract features,which ignore the dynamic attributes of input data,resulting in partial loss of useful information and not ideal retrieval effect.In order to solve this problem,a spatiotemporal information joint embedding-based end-to-end sketch-3D model retrieval method is proposed.Firstly,the sketch is represented as a dynamic drawing sequence,which reflects the temporal information contained in the drawing process;meanwhile,the 3D model is represented as a multi-view sequence to reflect the position rela-tionship between views.Secondly,an end-to-end dual-stream network including static spatial feature extraction and dynamic time series feature extraction is constructed.Combined with triplet central metric learning,a joint spatiotemporal feature embedding of cross domain data is established to fully capture the static and dynamic features contained in sketches and 3D models,and reduce the difference between cross domain.Finally,ex-periments are carried out on the standard public data sets SHREC2013 and SHREC2014.Compared with the existing work,the accuracy rate is higher,which verifies the feasibility and effectiveness of the proposed algorithm.