高级检索

基于通道干预渐进式差异减小网络的跨模态行人重识别

Progressive Difference Reduction Network with Channel Intervention for Visible-Infrared Re-Identification

  • 摘要: 可见光与红外图像的模态差异是导致跨模态行人重识别共享特征提取难度增大的本质原因。为降低两种模态的差异、提高行人重识别性能,本文提出了一种渐进式差异减小网络。首先,在可见光目标到红外图像集的检索识别阶段,根据因果推理理论设计了一个特定事实干预模块,通过通道变换生成的干预图像完成对可见光图像的干预,抑制可见光图像中的颜色信息干扰。其次,在红外目标到可见光图像集的检索识别阶段,设计了一个通道协调模块,将多通道的特征提取转换为单通道方式,使模型专注于学习可见光与红外两种图像的通道相关性。最后,为完成可见光和红外两种目标图像的相互检索识别,提出模态平衡损失方法,通过干预图像、可见光图像和红外图像,完成多个模态的平衡学习,进一步完成颜色特征抑制,补偿可见光图像在特定事实干预过程中的可鉴别丢失特征。仿真实验结果表明PDRNet相比于现有主流的跨模态行人重识别方法,在SYSU-MM01和RegDB两个标准数据集上均取得了较好的性能表现,rank1和mAP分别提高2个以上的百分点。模型源代码:https://pan.baidu.com/s/15LyVjCT-KEqaGoOc4nRlBw?pwd=0226。

     

    Abstract: The modality difference between visible light and infrared images is the fundamental reason which increases the difficulty of shared feature extraction for the visible-infrared re-identification. To reduce the difference between the two modalities and improve recognition performance, this paper proposes a progressive difference reduction network (PDRNet). Firstly, in the retrieval stage from visible light targets to infrared image sets, a fact-specific intervention module is designed based on causal reasoning theory. The intervention image generated through channel transformation is used to intervene in visible light images. It suppresses the color information interference in visible light images. Secondly, in the retrieval stage from infrared targets to visible light image sets, a channel coordination module is designed to convert multi-channel feature extraction into a single channel approach, focusing the model on learning the channel correlation between visible and infrared images. Finally, to achieve mutual retrieval between visible light and infrared target images, a modal balance loss is designed. By intervention images, visible light images, and infrared images, multiple modality balance learning is completed, further suppressing the color feature and compensating for identifiable lost features of visible light images during fact-specific intervention processes. The simulation experimental results show that PDRNet has achieved good performance on both SYSU-MM01 and RegDB benchmark datasets. Compared to existing the state-of-the-art methods, rank-1 and mAP has been improved by more than 2 percentage points, respectively. Model source code:https://pan.baidu.com/s/15LyVjCT-KEqaGoOc4nRlBw?pwd=0226.

     

/

返回文章
返回