Abstract: In image classification,visual separability between different object categories is highly uneven.However,existing deep Convolutional Neural Networks(CNNs)are trained as flat N-way classifiers,and few efforts have been made to leverage the hierarchical structure of categories.In this paper,we propose a deep multi-task learning algorithm to combine hierarchical classification with deep Convolutional Neural Networks,which makes the category hierarchy embedding into deep networks.Based on the structure of concept tree,we leverage the inter-task relatedness to learn more discriminative group-specific deep representations,and our deep multi-task learning algorithm can train more discriminative node classifiers for distinguishing the visually-similar atomic object classes effectively.Besides,we add an inter-level constraint to train the classifiers of non-leaf nodes,which can control inter-level error propagation.In the experiments,we build up two CNNs with different architectures,and our approach lowers the top-1 error by 2%~4%.