Abstract: Remote sensing image ship recognition is an important field of object identification, which has great application value in sea defense and rescue. However, the ship in remote sensing images is characterized by difficult identification caused by cloud and fog, land background interference and small size. To accurately identify ship objects in complex scenes, a visual attention mechanism is added to the feature extraction part of the network, which enhances the ability of the network to extract ship characteristic information. The learning method of multi-level feature extraction and de-quantitative operation is used to solve the problem of small ship size, and the learning strategy of hard example re-learning is used to weaken the interference of cloud mask and land background. Through the above methods, the comprehensive accuracy rate of ship identification reached 92.56%, the recall rate reached 89.26%, compared with other common object detection algorithms under the same experimental environment(PyTorch), the accuracy rate and recall rate have been significantly improved. Experimental results show that, to some extent, the proposed method solves the problem of ship segmentation and identification difficult in complex scenes. The code used in the experiment and some of the results are found in: https://github.com/curioyang/First_paper.