基于域适应与语义边缘的2D/3D配准方法
2D/3D Registration with Domain Adaptation and Semantic Edges
-
摘要: 2D/3D配准技术是脊柱手术机器人系统中的核心技术, 针对主流2D/3D配准方法受限于图像域间隙, 导致粗配准无法提供良好的初始位姿且精配准难以收敛的问题, 提出了一种基于域适应与语义边缘的双视图2D/3D配准方法. 首先, 为了降低真实X线片和模拟数据之间的域间隙, 利用块噪声对比损失和一致性损失训练了域适应变换网络用于将真实X线片的域分布迁移到模拟数据中; 然后, 训练了一种特征线提取变压器网络能够从2D图像中提取椎骨的语义边缘; 最后, 在配准阶段, 利用预适应变换网络降低图像域间隙,同时结合特征线提取网络引导双视图2D/3D配准过程,提高配准精度和效率. 在真实数据上的实验结果表明, 所提方法的平均目标配准误差为1.55 mm且配准成功率提高到了88.43%, 平均配准耗时为4.65 s, 满足了术中的实时性要求.Abstract: 2D/3D registration technology is a core technology in spinal surgical robotic systems. Addressing the limitations of mainstream 2D/3D registration methods constrained by the image domain gap, which often leads to suboptimal initial poses from coarse registration and convergence challenges in fine registration, this work introduces a novel dual-view 2D/3D registration approach leveraging domain adaptation and semantic edges. Initially, to mitigate the domain discrepancy between real X-rays and synthetic data, this work employs a contrastive learning for unpaired image-to-image translation network trained with patch noise contrastive estimation loss and consistency loss to transfer the domain distribution of real X-rays to that of synthetic data. Subsequently, a semantic edge extraction transformer is trained to extract semantic edges of vertebrae from 2D images. Finally, during the registration phase, the pre-adapted transformation network is utilized to reduce the image domain gap, concomitantly with the semantic edge extraction network to guide the dual-view 2D/3D registration process, thereby enhancing registration accuracy and efficiency. Experimental results on real data demonstrate that the proposed method achieves a mean target registration error of 1.55 mm and a registration success rate increased to 88.43%, with an average registration time of 4.65 s, thus satisfying the real-time requirements for intraoperative applications.