高级检索

变尺度特征融合与交叉训练的医学报告生成方法

Medical Report Generation Method Based on Multi-Scale Feature Fusion and Cross-Training

  • 摘要: 在对医学影像自动生成文本报告的过程中, 由于病灶尺寸小、形状不规则、训练数据量少等因素, 易导致影像报告出现误诊、漏诊问题, 提出变尺度特征融合与交叉训练的医学报告生成方法. 首先, 将条件全局池化后的粗粒度特征与随机丢弃后的细粒度特征相融合, 增强模型对不同尺度病灶的感知能力; 其次, 通过整体数据和局部细节双路交叉训练的策略, 间接丰富数据集, 增强模型的鲁棒性, 并在双路中分别使用通道分离思想, 进一步挖掘影像的通道信息; 最后, 通过多头注意力编解码网络, 得到准确的医学报告. 分别在IU-X-Ray和MIMIC-CXR数据集上与其他多种方法进行对比实验, 其中METEOR与BLEU-2分数分别提升了5.70%与3.13%. 结果表明, 所提方法可以有效地提升生成报告的可读性与准确性.

     

    Abstract: In the process of automatic report generation for medical images, due to the small sizes, irregular shapes of lesions, and small amount of training data, it was easy to lead to misdiagnosis and missed diagnosis in the reports. This paper proposes a medical report generation method based on multi-scale feature fusion and cross-training. Firstly, the method combines the coarse-grained features after conditional global pooling with the fine-grained features after random discarding to enhance the model’s perceptive ability to different lesion scales. Secondly, a two-way cross-training strategy through overall data and local details is used to indirectly enrich the dataset and improve the robustness of the model, while adapting channel separation principle to better mine channel information separately in the two ways. Finally, an accurate image report is obtained through the multi-head attention encoding and decoding network. Compared with other methods, the scores of METEOR and BLEU-2 are improved by 5.70% and 3.13% on the IU-X-Ray and MIMIC-CXR datasets, respectively. The results show that the proposed method can effectively improve the readability and accuracy of generated reports.

     

/

返回文章
返回