Dissertation > Excellent graduate degree dissertation topics show

The Research of Distributed Search Engine System Based on MPI

Author: ZhangJingYu
Tutor: LiangJiuZuo
School: Jiangnan University
Course: Computer Software and Theory
Keywords: Search engine Distributed Web crawling Chinese word segmentation Inverted index Load Balancing
CLC: TP391.3
Type: Master's thesis
Year: 2009
Downloads: 55
Quote: 0
Read: Download Dissertation

Abstract


The explosive growth along with the popularity of the Internet and network information people access to information is no longer rely on a limited range of sites to find , but rely on search engines to access vast amounts of data in the ocean of information . Search engine technology is the core technology has become essential for the development of the Internet , it 's increasingly important role . However , most of the existing search engines is centralized , has not well adapted to the further development of the network , distributed technology is the development trend of the next - generation search engine . In this paper, on the basis of analysis of the traditional search engine technology , distributed search engine system based on MPI (Message Passing Interface) . The system consists of parallel Web crawling and distributed index is composed of two parts . First, the detailed design and implementation of parallel to crawl pages , including the framework of its system , the main module running processes and URL scheduling algorithm . URL scheduling algorithm using a hash calculated , not only to achieve load balancing , and to some extent, to avoid a conflict . Then , by analyzing the important role of the index database in the timeliness and effectiveness of the search engine , a multi- process parallel sub- word indexing method . The Chinese Web database Corpus forward maximum matching method for Chinese word segmentation , with an efficient inverted index is stored index table . This method can accelerate the speed indexing and updated , and greatly improved space efficiency . Distributed Chinese search engine was erected on top of the MPI-based distributed network architecture using the MPI distributed nature , so that the search engines from centralized to distributed . The combination of static and dynamic task allocation strategies to improve the time and space efficiency and make the system easy to expand, create and update quickly crawl and index pages . The search engine more depth, more breadth to the information available to the user in the search on the Internet , more accurate, more rapid return to the user's query results .

Related Dissertations

  1. Study on Channel Allocation of Multi-Channel MAC Protocol in Ad-Hoc Network,TN929.5
  2. The Research of Fault-Tolerant Techniques for Parallel/Distributed Network Simulator PDNS,TP302.8
  3. Research and Design of One Kind of Paper’s QCS That Based on Embedded System,TP368.1
  4. Research of Communication Mechanism in the Distributed Network Based on Mobile Agent,TP393.02
  5. Web search engine related technology research,G354
  6. Research and Application about Ditributed Transaction Process Protocol,TP311.13
  7. Chain Enterprise Information Management System Design and Implementation of encryption technology,TP393.08
  8. Equipment management information system based on distributed three-tier application development and research,TP311.52
  9. Selection Algorithm in Heterogeneous Network Based on An Economic Model,TN929.5
  10. Research and Implementation of Key Techniques of Building Distributed System,TP338.8
  11. Research and Implement of Chinese Word Segment Techniques Based on the Conditional Random Field,TP391.1
  12. Research on Streaming Media Detection Methods Against DoS\DDoS Attack Based on Analysis of Self-similarity,TP393.08
  13. Research of Hybrid Application Layer Multicast Model Based on Load-balance,TP393.02
  14. Application of Multi-Agent Methods in Distributed Electric Power Generation Scheduling of Smart Grid,TM76;TM73
  15. The Research on Distribution Network Protection Strategy Based on the Analysis of Fault Characteristics of Inverter-based Distributed Generation,TM77
  16. Research on Approaches of the Subjective Automated Assessment,TP391.1
  17. A Secure Application Layer Multicast Group Member Management Model,TP393.08
  18. Research on Single-phase to Ground Fault Isolation Method of Distribution Network,TM862
  19. Based WebHarvest the Chinese financial news search engine design and implementation,TP311.52
  20. Chinese XML Compression Technology,TP311.11
  21. Based on distributed real-time database transaction scheduling policy Research and Improvement,TP311.13

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Retrieval machine
© 2012 www.DissertationTopic.Net  Mobile