3D Cross-Modal ConvFormer for Lung Cancer Recognition

Due to the irregular shape and large difference of lung tumors in 3D medical images, the feature extraction of lesions is insufficient and the recognition accurancy is not high, a 3D cross-modal lung tumor recognition model 3D-CConvFormer based on CNN and Transformer is proposed. Firstly, three Nets are utilized to learn the 3D PET, CT and PET/CT medical images. Secondly, a ConvFormer model is designed to fuse global features and shallow local features, and self-correcting convolution is utilized to effectively extend the receptive field to improve the extraction ability of lesion information in each modality. Finally, a dual-branch cross-modal feature interaction block with different resolutions is designed to interactively enhance cross-modal features and capture 3D multimodal detail information, The module can interactively enhance cross-modal features extracting ability using two global attention mechanisms to cross-learn different modal, global-local information. The 3D multimodal dataset of lung tumor is used in the experiments, with a total of 3 173 patients. Under the premise of better parameters and computation time, the accuracy of 89.25% and the AUC value of 88.74% are obtained by the 3D-CConvFormer model, which provided reliable computer-aided diagnosis for three-dimensional multi-mode lung tumor disease.