|
With the development of search technology , pure web search due to the shackles of IE , which showed low capacity applications , the equivalent of a desktop-based search client database application model , in particular, added a word search program function, which also makes the search function diversification , personalization becomes possible . Traditional sense of the search engines and the newly proposed concept of desktop search , the subject put forward a set of search engine from the client to the server implementation and realization . This scenario includes web spider , multi-threaded downloads , the Unicode file storage module , HTML / XML parser , word segmentation system , hashing secondary index database , Web Service the Pagerank ( page rank ) , word search program . The web spider Spider, responsible source of information on the various links in the network , including ordinary web information office resources , pictures, information and music video resources and flash animation, web spiders to the URL address and related information ( such as updating time , website ) deposited into the database , and all URL link record , call waiting behind PageRank module . Multi-threaded download the URL for the file is downloaded , with UNICODE renamed mechanism saved to the local hard disk. HTML / XML parser is responsible for already exists HTML document parsing , tick
|