2D/3D Registration with Domain Adaptation and Semantic Edges

2D/3D registration technology is a core technology in spinal surgical robotic systems. Addressing the limitations of mainstream 2D/3D registration methods constrained by the image domain gap, which often leads to suboptimal initial poses from coarse registration and convergence challenges in fine registration, this work introduces a novel dual-view 2D/3D registration approach leveraging domain adaptation and semantic edges. Initially, to mitigate the domain discrepancy between real X-rays and synthetic data, this work employs a contrastive learning for unpaired image-to-image translation network trained with patch noise contrastive estimation loss and consistency loss to transfer the domain distribution of real X-rays to that of synthetic data. Subsequently, a semantic edge extraction transformer is trained to extract semantic edges of vertebrae from 2D images. Finally, during the registration phase, the pre-adapted transformation network is utilized to reduce the image domain gap, concomitantly with the semantic edge extraction network to guide the dual-view 2D/3D registration process, thereby enhancing registration accuracy and efficiency. Experimental results on real data demonstrate that the proposed method achieves a mean target registration error of 1.55 mm and a registration success rate increased to 88.43%, with an average registration time of 4.65 s, thus satisfying the real-time requirements for intraoperative applications.