Dissertation > Excellent graduate degree dissertation topics show

Study on Hadoop-based Inverted Index

Author: DongChangChun
Tutor: LiXiaoGuang
School: Liaoning University
Course: Computer Software and Theory
Keywords: Hadoop Inverted index MIIS AMPS Multi- level index
CLC: TP391.3
Type: Master's thesis
Year: 2011
Downloads: 502
Quote: 1
Read: Download Dissertation

Abstract


With the arrival of the information age , every day a large number of new data are generated, how to quickly find the information needed to become the focus of the IT industry . Inverted index that can achieve fast retrieval tool is search engine information retrieval system most commonly used index structure . Meanwhile, the vast amounts of data in a distributed platform needs to be stored and processed , the corresponding inverted index file is also needed to run in a distributed platform, which is currently the most commonly used Hadoop as a platform for distributed systems , with a strong distributed storage and computing capabilities . Therefore , Hadoop platform for inverted index technology research has far-reaching significance. This article is for Hadoop platform, the inverted index techniques for the following research : 1 ) combined with HDFS file system supports only write once read , does not support any position to modify the characteristics , the paper design MIIS (Multi-level Inverted Index Structure ) inverted index structure , the inverted index stored in HDFS system, the inverted index for multi-level maintenance to support batch update files , but also can greatly improve query speed . 2 ) This document files with full consideration inverted index file association proposed AMPS (Align and Merge Placement Strategy) and a copy of the inverted index file placement policy , reducing the cost of communication between nodes , but also to achieve the retrieval keyword when can quickly locate the document in the local file's target . 3 ) combined with MIIS inverted index structure and AMPS inverted index and replica placement strategy, design of the inverted index construction algorithm , inverted index bulk add , delete, algorithms, and inverted index search algorithms , so that the inverted index in Hadoop platform can get better application . 4 ) build Hadoop clusters laboratory tests to verify the MIIS inverted index structure and AMPS placement strategy , improves search keywords and locate documents efficiency and reduce inter-node communication costs , speed up the bulk update speed.

Related Dissertations

  1. Research and Application of Map/Reduce Based Distributed Log Analyzer,TP311.52
  2. Design and Implementation of Online Shopping Prototype System Based on Hadoop,TP311.52
  3. The Research of Software Service Platform Based on Cloud Computing,TP311.52
  4. An Intrusion Detection System for High-Speed Networks,TP393.08
  5. Incremental Learning Method Based on Cloud Computing,TP311.13
  6. Hadoop-based video transcoding system design and implementation,TN919.81
  7. Fault Tolerance for MapReduce in the Cloud Environment,TP302.8
  8. Cloud-based mobile data storage backup system,TP309.3
  9. Publish subscribe model based design and implementation of middleware,TP311.52
  10. Cloud storage system for mass data,TP333
  11. Massive Video Conversion Platform Design and Implementation Based on Cloud Computing,TP311.52
  12. IaaS cloud computing - based Web application technology research,TP393.09
  13. Research on Technology of Massive Data Stores Based on Cloud Computing,TP333
  14. Research on the Key Techniques of Massive Image Data Management Based on Hadoop,TP751
  15. Research on Unified Access Plantform for Unstructured Data and Index Technology,TP311.52
  16. A Study on Compression Algorithm Performance Based Inverted Index,TP391.3
  17. Distributed Image Search Engine Design and Implementation,TP391.41
  18. Research and Implement of Job Scheduling Method for Multi_User MapReduce Clusters,TP311.13
  19. Design and Realization of Parallel File IO Based on Hadoop Distributed File System,TP338.6
  20. Website Crawler and Retrieval System Based on Lucene,TP393.092

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Retrieval machine
© 2012 www.DissertationTopic.Net  Mobile