Abstract: pen-based sketching tables are more likely to facilitate users' communication and creative interaction, avoiding the problems of high learning cost and cumbersome interaction interface that exist in traditional spreadsheet systems, compared to traditional spreadsheets systems based on the WIMP interface paradigm. However, the recognition of hand-drawn sketch tables still faces great challenges due to the lack of open source datasets dedicated to the recognition of hand-drawn sketch tables, as well as the ambiguity and abstraction of the sketches themselves and the arbitrariness of the users in drawing them. In order to solve the specific challenges such as the existence of complex cells, the overlapping of structural frame lines and content, and the redrawing and filling of handwriting, propose an algorithm for freehand sketch table recognition based on table structure understanding. A support vector machines (SVM) is used to classify the stroke information, and then the multi-peak detection algorithm is combined with the real intersection finding and cell feature attribute detection constructed in this paper to achieve the recognition of complex sketch table structure. The objective evaluation results show that the Tree-Edit-Distance-based Similarity (TEDS) of this algorithm in the sketch-oriented table structure recognition task is improved by more than 13% compared with the OCR baseline algorithm; the expert evaluation results show that this algorithm outperforms the OCR baseline algorithm in the evaluation dimensions of table structure recognition and matching the content to the corresponding cells.