Abstract: Traditional deep learning based colorization models may cause mistaken coloring in dealing with complex scenarios.For this problem,we proposed a pixel-wise generative adversarial network based colorization method.Firstly,we built a fully convolutional network for the generative model to deal with grayscale images of uncertainty scale.Moreover,the L1 loss between the output color maps and the real color components was calculated as the optimization goal.Secondly,we utilized a semantic segmentation network to build the discriminative model,of which a pixel-wise Softmax loss was calculated and propagated back to improve the performance of the colorization model for a better coloring output.Experimental results of color image quality comparison on Pascal Segmentation and ILSVRC2012 datasets show that the proposed colorization model achieves a higher accuracy and better discrimination between different objects compared with other colorization models.