Dissertation > Excellent graduate degree dissertation topics show

Design and Realization of Web Crawler with Hyperlink-based Algorithm

Author: ZhuJinTao
Tutor: HeFengLing
School: Jilin University
Course: Applied Computer Technology
Keywords: Crawler Search engine Hyperlinks Web URL
CLC: TP393.09
Type: Master's thesis
Year: 2007
Downloads: 201
Quote: 3
Read: Download Dissertation

Abstract


The rapid growth of information makes the discovery from the Internet and collect information resources has become particularly important in recent years on the Internet . A web crawler to find and save the page program from the network . Breadth-first search strategy is one of the common strategy of the web crawler , but ordinary width first search strategy often all pages have taken a non - discriminatory attitude , the crawling result has led to it is crawling back page quality is not high . This paper width -first search strategy carried out in-depth research , introduced a new width first search strategy - defined a new measure of web page hyperlink - based information search strategy , the distinction between the types of internal and external links the importance of the method , improved hyperlink - based search strategies to improve the quality of crawler pages ; final design a custom HTTP the web crawler system based on this strategy , and implementation using the following technologies : the class with the Web server to communicate ; conserve system resources according to the number you want to crawl the web dynamically determined the size of the thread pool .

Related Dissertations

  1. The Research and Implemention of Image Retrieval Based on User Interested Feature,TP391.41
  2. Research and Implementation of Mining Implicit User Interest,TP311.13
  3. Research on Web-Based Simulation Interoperability Testing Tool,TP391.9
  4. Study and Implementation on Human-computer Interaction Based on EoC System of NGB,TP11
  5. Cultural Industry Research in Shandong Province Set in Combination of Broadcasting Television Web, China Telecommunication Web and the Internet,G124
  6. Reptiles theme for Education News Design and Implementation,TP391.3
  7. University research project management system design and implementation,TP311.52
  8. Web-based video surveillance system architecture design and implementation,TP391.41
  9. CUDA-based regular expression matching system design and implementation,TP311.52
  10. Traditional manufacturing software, Web-oriented promotion of key technologies,TP311.52
  11. The RIA based Sivlerlight media playback and download engine design and implementation,TP311.52
  12. Process Engine based automated testing for Web Services Research,TP311.52
  13. Small and medium sized logistics enterprises Distribution Management System Design and Implementation,TP311.52
  14. Based WebHarvest the Chinese financial news search engine design and implementation,TP311.52
  15. Distributed search engine indexing and caching Security Strategy,TP333
  16. Embedded browser page displays Research and Implementation,TP368.1
  17. Research on Tacit Knowledge Transfer in Web-based Learning,G434
  18. The Design and Implementation of Electromagnetic Environment Monitoring System in Power Grid,TM76
  19. The Application of Web Data Mining Technology on Network Education BBS,G434
  20. OA System Design and Implementation of Jiangxi Modern Polytechnic College,TP311.52
  21. Research and Implementation of RFID Middleware Based on SOA,TP391.44

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Computer network > General issues > The application of computer network
© 2012 www.DissertationTopic.Net  Mobile