Abstract: Smoothness constraint is essential in addressing the ambiguity problem of stereo matching.An important issue involved in using this constraint is how to preserve discontinuity.In this paper,an adaptive smoothness term is introduced into stereo matching,where the smoothness penalty is changed according to local 2D structures.Firstly,we create a dataset composed of pairs of real color image and its corresponding depth map.Then,the conditional probability of continuity given the 2D structures is obtained through statistical analysis.Taking advantage of the conditional probability,we can calculate weight for smoothness penalty adaptive to local image content.Experimental results,implemented by applying this smoothness term into a classical graph cuts based stereo matching algorithm,demonstrate the effectiveness of our approach.