Abstract: A point matching algorithm based on shape context is proposed to solve the matching problems of feature points in infrared image and visible image.First,feature points are extracted by the curvature scale space corner detector.Second,the normal direction of each feature point on the curve is adopted as the main direction of the point,making the point descriptor rotation invariant.Third,the nearest feature point of each extracted one on the same edge is searched.And then,the histograms of edge pixels of the two key points regions are constructed to construct a 120-dimensional descriptor,and be normalized.Finally,the feature matching is realized via the nearest neighbor algorithm.Experiments show that the proposed algorithm can match feature points in both IR and visible images efficiently and correctly.