Abstract: In order to solve the problem of modal heterogeneity between visible light images and thermal infrared images due to different imaging principles, an asymmetric hierarchical feature fusion RGBT tracking network is proposed. Firstly, a two-stream network is used to extract visual light and thermal infrared features; then through the modal feature extraction module to mine different modal features and adaptively aggregate the obtained features to obtain features that are conducive to enhancing the visible light mode; finally, the aggregated features obtained by each layer and the visible light features obtained by the two-stream network perform enhanced fusion to obtain more robust features. Experimental results on GTOT, RGBT234 and LasHeR datasets show that the tracking precision rate (PR) and success rate (SR) of the network reach 92.2%/77.2%, 82.9% %/61.1% and 52.7%/40.3%, compared with the current mainstream RGBT target tracking network, both PR and SR have been improved, which verifies the effectiveness of the network.