Abstract: There is the possibility of object overlap and occlusion in the weakly supervised segmentation task, which makes it difficult for the pseudo-pixel mask generated by the seed region to accurately cover the object region, resulting in missed segmentation and wrong segmentation occur. In order to improve the above problems, a weakly supervised semantic segmentation network with multiple boundary references is proposed. The network obtains the object region boundary by cross-reference mode of spatial domain and frequency domain, so that provide a reasonable reference for the generation of pseudo-pixel mask by boundary information; on this basis, the pseudo-pixel mask generated can better match the object region. The evaluation results on the general dataset PASCAL VOC 2012 validation set and test set show that the proposed network mIoU reaches 68.7% and 69.2%. The experimental results proved that using multiple boundary information as reference can generate pseudo-pixel masks with better quality, effectively improve the problem of mismatch between the segmentation results and the object boundary, can improve the accuracy of weakly supervised semantic segmentation.