非对称的分层特征融合的RGBT跟踪网络
Asymmetric Hierarchical Feature Fusion Network for RGBT Tracking
-
摘要: 为了解决可见光图像和热红外图像由于成像原理不同而导致的模态存在异质性的问题, 提出了一个非对称的分层特征融合的RGBT跟踪网络. 首先通过双流网络来分别提取可见光和热红外的特征; 然后通过模态特征提取模块挖掘不同模态特征并对获得的特征进行自适应聚合以获得有利于增强可见光模态的特征; 最后各层获得的聚合特征与双流网络获得的可见光特征进行增强融合, 获得更具有鲁棒性的特征. 在GTOT, RGBT234和LasHeR数据集上的实验结果表明, 该网络的跟踪精度(PR)和成功率(SR)分别达到92.2%/77.2%, 82.9%/61.1%和52.7%/40.3%, 相较于目前主流的RGBT目标跟踪网络, PR和SR均有所提高, 验证了该网络的有效性.Abstract: In order to solve the problem of modal heterogeneity between visible light images and thermal infrared images due to different imaging principles, an asymmetric hierarchical feature fusion RGBT tracking network is proposed. Firstly, a two-stream network is used to extract visual light and thermal infrared features; then through the modal feature extraction module to mine different modal features and adaptively aggregate the obtained features to obtain features that are conducive to enhancing the visible light mode; finally, the aggregated features obtained by each layer and the visible light features obtained by the two-stream network perform enhanced fusion to obtain more robust features. Experimental results on GTOT, RGBT234 and LasHeR datasets show that the tracking precision rate (PR) and success rate (SR) of the network reach 92.2%/77.2%, 82.9% %/61.1% and 52.7%/40.3%, compared with the current mainstream RGBT target tracking network, both PR and SR have been improved, which verifies the effectiveness of the network.