高级检索

基于Mamba 的多模态颞下颌关节疾病自动诊断方法

Heterogeneous Multi-modal Mamba for Automatic Temporomandibular Joint Disease Diagnosis

  • 摘要: 当前,颞下颌关节紊乱病(temporomandibular joint disorder,TMD)的临床诊断主要依赖牙科医生对核磁共振成像(magnetic resonance imaging,MRI)的人工判读,存在主观性强、效率低等问题。尽管深度学习技术已被引入TMD自动诊断任务,但现有方法多局限于单一影像模态,忽略了非影像信息(如临床表格数据)中所蕴含的关键诊断先验,且在建模MRI影像全局长程依赖关系方面仍存在不足。为此,提出一种基于Mamba的多模态颞下颌关节疾病自动诊断方法,融合MRI影像与临床表格两类数据,以提升诊断的准确性与泛化能力。首先,基于Mamba的图像编码器采用级联双分支状态空间块(dual-branch state space block,DSSB),同时捕捉MRI影像的局部细节与长程全局上下文,实现对图像判别特征的充分表征;并基于BERT的文本编码器处理临床表格数据,提取患者个体化诊断特征;随后,文本引导注意力融合(text-guided attention fusion,TAF)模块利用最优传输注意力机制对这些互补特征进行融合,实现有效的多模态交互。最后,将融合后的多模态特征输入预测层,实现对3种病变类型的自动分类输出。在真实临床TMD数据集上的实验结果表明,所提方法的曲线下面积为0.918,准确率为0.780,F1分数为0.774,均显著优于现有单模态及多模态诊断方法。

     

    Abstract: Current clinical diagnosis of temporomandibular joint disorder (TMD) mainly relies on the manual interpretation of magnetic resonance imaging (MRI) results by dentists, which suffers from high subjectivity and low efficiency. Although deep learning technologies have been applied to the automatic diagnosis of TMD, most existing methods are limited to single-modal imaging data. They ignore the critical diagnostic prior knowledge contained in non-imaging information (i.e., clinical tabular data) and fail to effectively model the long-range global dependencies of MRI images. To address these issues, this paper proposes a Mamba-based multi-modal automatic diagnosis method for temporomandibular joint diseases, which integrates MRI images and clinical tabular data to improve the accuracy and generalization ability of diagnosis. Firstly, a Mamba-based image encoder adopts cascaded dual-branch state space blocks (DSSB) to simultaneously capture local details and long-range global context of MRI images, so as to fully extract discriminative image features. Meanwhile, a BERT-based text encoder processes clinical tabular data to acquire personalized diagnostic features of patients. Subsequently, a text-guided attention fusion (TAF) module leverages the optimal transport attention mechanism to fuse the above complementary features and realize effective multi-modal feature interaction. Finally, the fused multi-modal features are input into the prediction layer to achieve automatic classification of three types of lesions. Experimental results on a real clinical TMD dataset show that the proposed method achieves an area under the curve (AUC) of 0.918, an accuracy of 0.780, and an F1-score of 0.774, which are significantly superior to those of existing single-modal and multi-modal diagnosis methods.

     

/

返回文章
返回