Dissertation > Excellent graduate degree dissertation topics show

Research on Data Cube Technology Based on MapReduce

Author: ChenLin
Tutor: YangKeHua; ChenYiHui
School: Hunan University
Course: Computer technology
Keywords: OLAP Hierarchical Cluster OLAP Query Optimization Hadoop MapReduce
CLC: TP311.13
Type: Master's thesis
Year: 2013
Downloads: 1
Quote: 0
Read: Download Dissertation

Abstract


With the rapid development of internet and information technology industry, theamount of data generated by network is ever-growing. Large data contains moreuseful information, it also brings more challenges. On-Line Analytical Processing(OLAP) as an important technology for data storage and analysis is also facing thechallenge of huge amount of computation. As the primary means of OLAP, how toefficiently deal with the massive data of the Data Cube is a key issue in both researchand application field of OLAP. Google’s MapReduce is a simplified distributedprogramming model for processing large-scale data. Based on this distributed parallelmodel, this thesis presents parallel clustering, update and query of the Data Cube. Themain research achievements and innovative points are as follows:(1) Parallel Clustering of Data Cube: Based on the equivalence relation betweenthe semantic features and multi-dimensions of Data Cube, a parallel semantic Cubehierarchical clustering algorithm based on the MapReduce framework is proposed.The Data Cube can rapid clustering, and ultimately save the equivalence classes of theupper and lower bounds to realize the compressed storage of Data Cube. This methodcan effectively save storage space, and speed up clustering procedure as well. Whencluster information and hierarchical information are saved, it can also provide theadvantage of rapidly updates of Data Cube and the possibility of analysis of OLAPquery behavior.(2) Incremental Maintenance of Data Cube’s hierarchical clustering: based onData Cube equivalence class, combining with the hierarchical relations between theequivalence classes, an efficient batch update algorithm of Data Cube in theMapReudce parallel framework is proposed, in this way, the problem of lowefficiency which is caused by large amount of data maintenance is solved.(3) Parallel OLAP queries: based on Data Cube equivalence classes, some paralleloptimizations on OLAP query point and query range are realized. What’s more, in theimproved MapReudce model, a cache-based OLAP query optimization algorithm isproposed. By defining various operations in the OLAP query, multiple OLAP queriesis parallel processing, which greatly improves the query efficiency.This thesis also analysis various operations of the semantic Cube parallelization indetails. The implementations of these operations under the MapReduce model are properly designed. Some comparisons of parallel algorithm and traditional algorithmare made to prove the superiority of parallel algorithm.

Related Dissertations

  1. Research and Application of Home-School Education Platform Interactive Based on Warehouse,TP311.13
  2. Government Decision Support System of Industrial Economics,TP311.13
  3. Design and Implementation of Network Teaching and Research OLAP Analysis System Based on Data Warehouse,TP311.13
  4. Design and Implementation for Decision Support System of Drug Administration Based on Data Warehouse,TP311.13
  5. The Research of Data Mining and OLAP about Medical Insurance in Social Protection System,TP311.13
  6. Research and Design of Equipment Material Support Integrated Management System Based on Data Warehouse,TP311.13
  7. Design and Implementation of Online Shopping Prototype System Based on Hadoop,TP311.52
  8. Design of the Mobile Learning System Based on Hadoop,G434
  9. The Research and Application of Multi-dimensional Association Rules Algorithm Based on Mondrian,TP311.13
  10. Construction and quick access to the marine disaster prevention and mitigation technology research data warehouse,TP311.13
  11. Construction and Application of Computer Audit Based on Date Warehouse and Date Mining,TP311.13
  12. PG steelworks MES system design and development of data mining,TP311.13
  13. RCCs data warehouse system design and implementation,TP311.13
  14. OLAP Query Optimization Research and Implementation,TP311.13
  15. The Study on Evaluation to Efficiency of R&D Input-output in China Based on DEA Model,G322
  16. 3D Mannequins Generating Engine Based on eMTM with MapReduce,TP391.41
  17. The Research of Software Service Platform Based on Cloud Computing,TP311.52
  18. Research on Fast Queryalgorithm of Massive Data,TP311.13
  19. An Intrusion Detection System for High-Speed Networks,TP393.08
  20. Incremental Learning Method Based on Cloud Computing,TP311.13
  21. Research on Query Optimization in Dameng OLAP,TP311.13

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer software > Program design,software engineering > Programming > Database theory and systems
© 2012 www.DissertationTopic.Net  Mobile