Abstract: To develop real time and robust travel aids system for the blind, this paper introduces an adaptive scene segmentation and obstacle detection algorithm based on RGB-D data.Firstly, a roughto-fine plane segmentation algorithm based on depth data is used, which makes use of the indoor structure information.Secondly, agraph-based scene segmentation algorithm is introduced, which adaptively combine the results of plane segmentation and RGB image segmentation.Thirdly, a multilevel object detection strategy is applied to recognize segmentations as ground, desk, wall or other obstacle.Finally, a simplified 3D reconstruction of the scene is used to demonstrate the analysis results.The experimental results show the proposed method effectively improves performance of scene segmentation and obstacle detection, which well combines depth data and geometry structural information of indoor scene.The proposed algorithm is fast and robust, which can be applied to indoor obstacle avoidance system for the blind.