Infrared-Visible Person Re-identification Via Multi-modality Feature Fusion and Self-distillation

Most of the existing cross-modality person re-identification methods mine modality-invariant features, while ignoring the discriminative self-owned features in different modalities. In order to fully mine the born features in different modalities, an infrared-visible person re-identification method via multi-modality feature fusion and self-distillation is proposed. Specifically, we propose an attention fusion mechanism based on a dual classifier, which assigns a larger fusion weight to the self-owned features of each modality, and conversely, a smaller weight to the common features, so as to obtain the multi-modality fusion features containing each modality’s discriminative owned features. At the same time, in order to improve the robustness of the features extracted by the network and adapt to the changes of pedestrian appearance, a memory storage is constructed to store the multi-view features of pedestrians. In addition, a parameter-free dynamic guidance strategy for self-distillation is designed. Under the guidance of multi-modality fusion features and multi-view features, this strategy is used to dynamically strengthen the multi-modality and multi-view reasoning capabilities of the network. Finally, the network is able to infer the features of pedestrians with different views of another modality from the single-modality image of a pedestrian, thus, improving the performance of the model for cross-modality person re-identification. Based on the PyTorch deep learning framework, Rank-1 reaches 63.12% and 92.55% respectively on the public dataset SYSU-MM01 and RegDB, and mAP reaches 61.51% and 89.55% respectively. The experimental results prove that the proposed method is better than the comparison methods.