Abstract: In the conventional rate distortion optimization（RDO） video coding, the measure of distortion is mainly from the perspective of signal processing, while dose not fully take into account the characteristics of visual perception. In this paper, in order to establish a RDO model which is more consistent with the human visual perception, we introduce the structural similarity and the content saliency information into the distortion metric. An adaptive Lagrange multiplier selection scheme is presented to allocate the bit resources more rationally by keeping the balance of the bit-rate and the visual quality. Experimental results show that the proposed method can reduce bit-rate effectively under equal visual quality.