高级检索

基于多模态多实例学习的胃癌患者生存预测算法

Survival Prediction Algorithm for Gastric Cancer Patients Based on Multi-Modal Multi-Instance Learning

  • 摘要: 生存预测对于胃癌患者的治疗具有重要意义.针对传统基于组织病理图像的生存预测算法存在像素级标签缺失、信息量大以及模态单一等问题,提出一种基于多模态多实例学习的胃癌患者生存预测算法.首先使用多层感知器和自监督学习方法SimCLR分别提取临床数据和组织病理图像特征,然后采用基于全局感知的多实例学习方法提取高分辨率下的包级嵌入,使用平均池化方法得到低分辨率下的组织病理图像实例级嵌入,最后通过多模态融合方法将包级嵌入、实例级嵌入和临床数据特征进行融合,以实现不同模态数据之间的信息交互和不同放大倍数下图像信息的充分利用.在云南省肿瘤医院胃癌患者病理数据库上的实验结果表明,与传统的多实例学习方法相比,所提算法在5×和20×组织病理图像下胃癌患者生存预测的准确率分别提高了5.6~10.0个百分点,4.1~7.5个百分点,与常规的多模态融合方法相比,提高了3.5~4.9个百分点.

     

    Abstract: Survival prediction is vital for treating gastric cancer patients. To overcome the shortcomings of classic survival prediction algorithms based on histopathology images, such as pixel-level label missing, high information volume, and single modality, a survival prediction algorithm for gastric cancer patients based on multi-modal multi-instance learning is proposed. Firstly, hidden features behind the clinical data and histopathological image are extracted using MLP and self-supervised learning method SimCLR, respectively. Subsequently, a globally-aware multi-instance learning approach is employed to extract bag-level embeddings at high resolution and instance-level embeddings of histopathological images at low resolution are obtained using an average pooling method. Finally, a multi-modal fusion approach is utilized to integrate bag-level embeddings, instance-level embeddings, and clinical data features, thus facilitating information interaction between different modalities and effective utilization of image information at various resolutions. The experimental results on the pathological database of gastric cancer patients from Yunnan Cancer Hospital show that, compared to traditional multi-instance learning methods, the proposed algorithm improves the prediction accuracy by 5.6-10.0 percentage points and 4.1-7.5 percentage points from 5×and 20×histopathological images, respectively. Furthermore, compared to conventional multi-modal fusion methods, the proposed algorithm exhibits an accuracy improvement of 3.5-4.9 percentage points.

     

/

返回文章
返回