Forecast and Filter Method for Web Page Gathering Based on Latent Sematic Indexing

Following rapid expansion of huge information on Web, the efficient Web information gathering on specified fields becomes a key topic in information retrieval research. Based on LSI (Latent Semantic Indexing), this paper presents the forecast and filter method for Web page gathering. The method applies LSI on Web page sets provided by user to design the interested model and compute the relevance between Web page and the interested model for text filtering. Based on the analysis of Website structure, forecast for the relevance of Web page controls the gathering process. As a result, gathering time is shortened, storage decreased, retrieval speeded, and net resources are saved.