Abstract: To address the problem of low accuracy of minority clothing image retrieval that lacks semantic labels and local feature complexity for minority clothing,a clothing image retrieval method using label optimization and semantic segmentation is proposed for various minority clothing.Firstly,a probabilistic model based on general semantic labels and minority semantic labels was constructed to analyze the visual style and optimize the defined semantic labels for minority clothing.Secondly,a side-branch network and a fully-connected CRF were added to the basis of the full convolutional network structure.Then,combining the training image with annotation pairs and optimized semantic labels,the query image was semantically segmented.Finally,a deep supervised hashing algorithm based on multi-task was used to encode the semantic segmentation result to a binary code,and the retrieval results of minority clothing image were output by similarity calculation.The proposed method is validated on the dataset of minority clothing.Experimental results show that the proposed method can effectively improve the accuracy of semantic segmentation and minority clothing image retrieval.