Dissertation > Excellent graduate degree dissertation topics show

Design and Implementation of Distributed Query Algorithm Processing Communication Data Based on Hadoop

Author: ChenYong
Tutor: LinYouFang
School: Beijing Jiaotong University
Course: Computer Science and Technology
Keywords: Hadoop platform Map/Reduce algorithm Distributed query Hbase
CLC: TP311.133.1
Type: Master's thesis
Year: 2009
Downloads: 1786
Quote: 30
Read: Download Dissertation

Abstract


Data is the carrier of information. With the development of information technology, data in the modern social life assumes an increasingly important role. Social network analysis distils useful information from the social network data by graph theory, data mining techniques and so on. Usually of the data set is very large. So the ability of data processing should be excellent.Large-scale social network communications data analysis and visualization system is a social network analysis tools dealing specifically with communication data set. For the system, Expansion of the hierarchical data involves querying in extensive dataset and thus it needs a high efficiency of data query requirements. Using the traditional relational database such as Oracle or SQL Server can meet the complex conditions while the inquiry, but when dealing with TB-class large-scale data sets, it’s unable to do as much as we would like to. At the same time, BFS algorithm, which has traversal operation, is is very low in relational databases.In such a case, we need to solve data query and processing bottlenecks exist. After analyzing the existing distributed storage systems and cloud computing platform, we choose Hadoop platform for distributed data storage and query to improve the program.The paper focus on the communication data distributed storage and query based on Hadoop platform. It tells how to design the Hbase-based communication data model of social network data. We implement the conditions query, design and optimize data model. Finally, the clients can access services from Haoop platform. We also design and implement Map/Reduce algorithm for communication data set. Map and Reduce functions implement the data parallel query processing. In the data query process, the traverse process is put in the Reduce function, so that the BFS algorithm traverse can also run in parallel. This is in large measure to optimize the data query and the efficiency of stratification expansion.The implementation of communication data distributed storage and query based on Hadoop platform has very important significance. Hadoop platform needs to be deployed only in the ordinary, cheap PC to run, but have high efficient to deal with data, it has high value and application of practical significance.

Related Dissertations

  1. Design and Implementation of Online Shopping Prototype System Based on Hadoop,TP311.52
  2. Research and Implementation on a Distributed Service Registry Based on HADOOP Platform,TP393.09
  3. Research of Adaptive Local Hyperplane K-nearest Neighbor Classification Algorithm Based on Hadoop Platform,TP301.6
  4. The Research and Implementation of the Chinese Character Recognition Based on MapReduce,TP391.41
  5. A Study on Reflective Hydro-Database Access Middleware,TV21
  6. Mobile Agent in Distributed Database Query Application,TP311.13
  7. Distributed Parallel Database System DPSQL Distributed Queries and Distributed Transactions in the design and implementation,TP311.13
  8. The Research and Application of Distributed Data Query Based on Mobile,TP311.5
  9. Research and Implementation of the Liaohe Oilfield distributed personnel MIS system,TP311.52
  10. Research on Scientific Data Grid Distributed Query Architecture and Its Key Technologies,TP393.01
  11. Distributed database query optimization techniques,TP311.13
  12. Research and Implementation of the digital organism database distributed query optimization , and distributed transaction processing,TP311.13
  13. Research and Implementation on HBASE Based Column-oriented Compression Algorithms,TP311.13
  14. Research and Application of Distributed Social Network Analysis Support System,TP393.09
  15. Research and Design of RDF Storage System Based on HBase,TP333
  16. Parallel Clustering Algorithm Based on MapReduce,TP311.13
  17. Research on Query Optimization of Distributed Database,TP311.138
  18. Anytime Query Processing in P2P Networks,TP393.02
  19. A Distributed Index Research Based on B~+-Tree in Parallel Data Warehouses,TP311.133
  20. Research and Design for Data Synchronization on Heterogeneous Distributed Database,TP311.133.1
  21. The model structure and query optimization of distributed database for read-only applications,TP311.133.
  22. Research and Development of the Expressway Toll Collection Network System and Distributed Database,TP311.133.1

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer software > Program design,software engineering > Programming > Database theory and systems > Distributed database
© 2012 www.DissertationTopic.Net  Mobile