Abstract: In order to reduce the impact of loss of spatial information generated by pooling operator and improve the performance of transfer learning for weakly-supervised semantic segmentation algorithm with deep convolutional neural network, this paper designs a weakly-supervised image semantic segmentation algorithm based on multi-model ensemble. Based on transfer learning algorithm, the method firstly utilizes the semantic features from last convolutional layer of a multi-scale image and the convolutional features from the middle and deep layers of a single-scale image to respectively train two different homogeneous segmentation models. And then these models are weighted integrating with the original transfer-learning model to get the final segmentation model. In addition, the algorithm combines the confidence of categories to adjust the pixels’ confidence expecting to suppress the false positive regions in the segmented image to improve the accuracy. Finally, the proposed algorithm is tested in challenging VOC2012 dataset. The results show that the mean intersection-over-union of the proposed algorithm is 55.3% on validation dataset and 56.9% on test set, outperforming the original transfer-learning algorithm by 6.1% and 11.1%, respectively. And the method performs favorably against other segmentation methods using weakly-supervised information based on class labels as well.