Abstract: In order to bridge the semantic gap in image retrieval, this paper proposes an approach to annotate image automatically by modeling continuous visual features directly. Firstly, we extend probabilistic latent semantic analysis (PLSA) to model continuous quantity. In addition, corresponding Expectation-Maximization algorithm is derived to determine the model parameters. Secondly, in terms of the characteristics of different modalities, we present a semantic annotation model which employs continuous PLSA and standard PLSA to model visual features and textual words respectively. The model learns the correlation between these two modalities by an asymmetric learning approach and then it can predict semantic annotation precisely for unseen images. Finally, we conduct experiments on a standard Corel dataset consisting of 5 000 images. In comparison to several state-of-the-art approaches, our approach can achieve higher accuracy.