Advanced Search
Chaoyi Wang, Nannan Yu, Yu Qiao, jiankang ren, Dongsheng Zhou, xiaopeng wei, qiang zhang, Xin Yang. Event-Based Image Semantic Segmentation[J]. Journal of Computer-Aided Design & Computer Graphics. DOI: 10.3724/SP.J.1089.2023-00698
Citation: Chaoyi Wang, Nannan Yu, Yu Qiao, jiankang ren, Dongsheng Zhou, xiaopeng wei, qiang zhang, Xin Yang. Event-Based Image Semantic Segmentation[J]. Journal of Computer-Aided Design & Computer Graphics. DOI: 10.3724/SP.J.1089.2023-00698

Event-Based Image Semantic Segmentation

  • The image semantic segmentation task is essential for computer vision, and it is applied to various fields such as obstacle avoidance for drones, autonomous driving, medical imaging and so on. Although it has been developed based on RGB cameras, there are still some significant challenges that cannot be ignore in actual scenes. For example, it is difficult for RGB cameras to image effectively in over-exposure and low-light scenes and cannot provide sufficient semantic information. RGB cameras will leads motion blur where fast-moving objects in the scene due to the sampling method, pose challenges for semantic segmentation task. Event camera, a novel bionic vision sensor, is different from the imaging principle of traditional RGB cameras, which captures changes in the light intensity of pixels and generates event data asynchronously. With the advantages of high dynamic range, high response speed, and low power consumption, event camera can image effectively in challenging scenes such as overexposure and low light without motion blur. Therefore, it can provide a new solution for semantic segmentation tasks in realistic challenging scenarios. However, there is lacking semantic segmentation datasets based on event cameras, and it takes a lot of manpower and material resources to create an image semantic segmentation dataset with high-quality annotations. To solve this, existing event-based semantic segmentation datasets utilize deep learning algorithms to predict results as semantic labels. The pseudo-class label is not accurate enough due to the performance of the algorithm. To solve these above problems, this paper creates a large-scale event camera image semantic segmentation dataset named Carla-Semantic, which provides synchronized RGB images, event frame images, and accurate pixel-level semantic labels for image semantic segmentation tasks. In addition, considering the uneven distribution and the sparseness of event data in some areas, we design an event-based image semantic segmentation network named EVISS, which can further enhance event feature representation from a global perspective and strengthen the global connection of each point in the image. With the improved graph Laplacian formula, we introduce the diagonal matrix of the position-independent attention mechanism, which can capture the long-distance context relationship better, so as to extract high-level event features better. We make experiments on the Carla-Semantic dataset to evaluate our method. Compared to Ev-SegNet, the proposed method in this paper achieves an improvement of 1.85% in the MPA metric and 1.68% in the mIoU metric.
  • loading

Catalog

    Turn off MathJax
    Article Contents

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return