Image Colorization via Object Scale Adaptive Transformer

The existing image colorization methods based on Transformer solely focus on global contextual modeling of images and have less investigation on the validness of object scale information in image scenes. We propose an object scale adaptive Transformer-based grayscale image colorization method by constructing an end-to-end trained encoder-decoder network architecture. At the encoder stage, we adopt object scale adaptive Transformer blocks that use a scaling factor adaptive selection module to capture the object scale information and compute the reduction ratio of the attention layers. This allows Transformer blocks to compute object-scale adaptive attention, facilitating global modeling of objects with different scales while reducing computational complexity. At the decoder stage, we add a semantic-aware module to strengthen the semantic constraints of decoding features, which enable the model to understand object sematic well, avoiding the color bleeding and color ambiguity artifacts. Extensive experiments on ImageNet and COCO-Stuff datasets demonstrate that our method can generate more vivid and semantically matched color images compared with state-of-the-art methods.