Dissertation > Excellent graduate degree dissertation topics show

The Research on Focused Crawling Algorithm in Vertical Search Engine

Author: ChenKeQin
Tutor: TanJunShan
School: Central South University of Forestry Science and Technology
Course: Applied Computer Technology
Keywords: vertical search engine topical/focused crawler interclass linkage Baseline focused crawler tunneling
CLC: TP391.3
Type: Master's thesis
Year: 2009
Downloads: 147
Quote: 0
Read: Download Dissertation

Abstract


With the ever-expanding information, people become increasingly dependent on vertical search engines. Surrounding the research on this hotspot, the important part of the topic-specific search engine that is called topical crawler is discussed in this paper. Focused crawling is able to crawl particular topical portions of the world wide web quickly without having to explore all web pages. And now, it is more and more widely applied in the fields of topic-specific search engines and analyzing site structure, and so on.We introduce the structure of rules-based crawler which can download "important" web pages first by counting the sum of URL important value and text relativity value. We give an algorithm we designed. This crawler uses the search algorithm both based on Web hyperlink structure and page content. In the course of the relativity judging between the page content and the topic, the method based on vector space model which is widely applied in the field of the text classification is used. Besides, the influence to weight of anchor text and text locality are taking into account. In the course of the relativity judging between the URL content and the topic, we develop a new method that statistically capture linkage relationships and extract rules among the classes (topics), then guide our focused crawler by using these rules. Meanwhile it has an improvement on Page Rank algorithm.In our work, we started with a focused-crawling approach designed by Soumen Chakrabarti, Martin van den Berg and Byron Dom, called baseline crawler. Building on this crawler, we developed a rule-based crawler, which uses simple rules derived from interclass (topic) linkage patterns to decide its next move.Initial performance results show that this rule-based Web-crawling approach uses linkage statistics among topics to improve a baseline focused crawler’s harvest rate. This rule-based crawler also enhances the baseline crawler by supporting tunneling.

Related Dissertations

  1. Research on the Theory of the Transmission-line Metamaterials and Their Applications,O441.6
  2. Research on Dust-Isolation Technology with High Pressure Air Curtain for Mechanized Tunneling Face,TD714.4
  3. Empirical research on the behavior of the controlling shareholder of the quality of accounting earnings,F832.51;F224
  4. Research on Crawling Techniques of Focused Search Engine,TP391.3
  5. Research on the Structure and Material of the Composite Tunnel Pavement,U416.217
  6. Theoretical Study on Quantum Transport Properties of Multi-Treminal Coupled-Quantum-Dot Systems,O471.1
  7. Soliton and Landau-Zener Tunneling Dynamics of Bose-Einstein Condensates Trapped in optical Lattices,O431.2
  8. Personalized Multi-media Resources to A Vertical Search Engine Technology Research,TP391.3
  9. Research on the PLC Control System of the Roadheader Robot.,TD421.5
  10. Research on and Realization of a Vertical Search Engine,TP391.3
  11. Research and Implementation of Vertical Search Engine Techniques,TP391.3
  12. Research on Domain-Oriented Search Engines Based on User Behavior,TP391.3
  13. Improvement on Endurance Performance of Embedded Super Flash Memory with SST 0.25μm Technology,TP333
  14. Pyramid Structure、Tunnelling and Company Performance,F272
  15. Research and Implementation on Trading Information Oriented Vertical Search Engine,TP391.3
  16. Vertical position based on Lucene Search Engine Research and Implementation,TP391.3
  17. The shield cutter rotary drive hydraulic system modeling and simulation research,U455.31
  18. Adsorption Behavior Research of Iron Phthalocyanine on Ag(110) Surface by STM,O485
  19. A Study on the Major Shareholder’s Behavior and the Performance of Seasoned Equity Offerings,F276.6;F224
  20. Debt covenants with the controlling shareholder tunneling Empirical Study,F276.6

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Retrieval machine
© 2012 www.DissertationTopic.Net  Mobile