Abstract: Aiming at a series of problems such as self-intersection and holes in the mesh data obtained by the existing oral scanning equipment, a fast and effective self-intersection detection algorithm is proposed. To reduce the time complexity of global mesh self-intersection detection, a special octree structure is imported, with a subdivision strategy, the triangular facets are placed in different cubes to detect self-intersection and eliminate incorrect linkages by deleting operations. However, this will bring a lot of holes. To remedy these holes, a special bilateral data structure is designed, which can quickly and effectively detect single-connected closed holes and correctly process vertices shared by multiple holes. After remedying the holes, the Laplacian smoothing strategy is used to process the hole area, and achieve an effect that the hole is filled with natural and smooth transition. Experiments on the mouth scan data show that the proposed algorithm can not only rapidly detect the self-intersecting region of mesh data, but also efficiently complete the hole filling operation. Compared with the existing algorithm, the processing speed is about 10 times faster on the data with more than 1 million meshes.