Dissertation > Excellent graduate degree dissertation topics show

Performance Optimization and Applications of MapReduce in Cloud Computing

Author: ChenXiangXiang
Tutor: WuKaiGui
School: Chongqing University
Course: Computer System Architecture
Keywords: Cloud computing Huge amounts of data MapReduce Programming model Classifier
CLC: TP3
Type: Master's thesis
Year: 2011
Downloads: 718
Quote: 3
Read: Download Dissertation

Abstract


Since 2007, Cloud computing has gradually become more popular concept of the international IT community, along with the surge in the amount of data, how fast and efficient storage and computing massive data become the urgent need to address the problem of the scientific community, and this kind of problem is precisely cloud computing to launch one of the engines, the popularization and application of cloud computing has become the industry can not be avoided and the reversal of the trend. But cloud computing itself says, it is only a mode of thinking, if we really want to play to its strengths, in addition to the hardware is required, it is more important to have the support and cloud computing programming model of thought, and Google's MapReduce parallel programming model, with its simple and powerful interface allows parallel processing becomes simple, provides software support for the calculation of mass data in cloud computing. This paper analyzes the concept of of Google MapReduce its underlying file storage system GFS, advantages and implementation mechanism. And intermediate results data for the process of the implementation of the MapReduce processing mechanism is not flexible, intermediate results is not the first time to reduce the number of shortcomings, the introduction of associative array in the MapRedeuce map function, you can make intermediate results of the merge operation in the Map function automatically, more effective in reducing the number of intermediate results, reduce the burden on the network, thereby enhancing the efficiency of the system. In this paper, to improve MapReduce is designed and implemented based MapRedeuce text classifier. , Massive data classification problem is often encountered in the field of text processing and data mining, the traditional algorithm, however, can only be adapted to the small-scale data, the execution speed of the algorithm is getting slower and slower as the amount of data increases, real-time getting worse, become the bottleneck of traditional data mining. This new classifier construction method in a cluster in parallel to achieve the classification of building, greatly improving the efficiency of the algorithm has better real-time. In order to verify the performance the MapReduce improved after using MapReduce open source Hadoop to experiment, as a measure of algorithm running time, the results show the efficiency of the new algorithm than traditional algorithm to be much higher. For the realization of the classifier, we are also in the Hadoop platform, we can see, the adoption of the outcome by contrast has better efficiency and scalability of MapReduce-based classifier.

Related Dissertations

  1. ISAR Imaging Simulation of Space Targets and Target Recognition Based on ISAR Images,TN957.52
  2. The Research of Dynamic Trust Model on Cloud Computing Platform,TP309
  3. The Research of Land-Use Optimization Approach Based on Non-Point Source Pollution Controlled,X24
  4. Cloud-based digital library service model,G250.76
  5. Design and Implementation of Online Shopping Prototype System Based on Hadoop,TP311.52
  6. Based on the study of resource management in the cloud environment of credibility,TP315
  7. Design and Implementation based the Google platform promotional modules commodity module,TP311.52
  8. Research and Implementation of Monocular Vision-Based Vehicle Detection Algorithm,TP274
  9. High-performed Kernel Classification Methods Based on Multi-kernel Learning,TP391.41
  10. The Operation and Maintenance of ITIL Based on Cloud Computing,TP311.52
  11. The Research of Software Service Platform Based on Cloud Computing,TP311.52
  12. Research on Diagnosis Methods of Breast Masses Based on Reference Images,TP391.41
  13. Virus Detection Technology Based on Artificial Immune,TP393.08
  14. Web application system design based on Google's cloud computing platform and,TP393.09
  15. Data Privacy-Preserving Schemes for Cloud Computing,TP393.08
  16. The Research and Realization of Unwanted Code Monitoring System Based on Heuristic Algorithm,TP393.08
  17. An Intrusion Detection System for High-Speed Networks,TP393.08
  18. Algorithm for the Traffic Flow Prediction Based on Improved Non-Parametric Regression Method,F570
  19. The Design and Implement of Cloud Storage System Client Based on Hadoop,TP333
  20. Incremental Learning Method Based on Cloud Computing,TP311.13
  21. Research of Resource Provisioning Technique in Telecom Business Supporting Based on Cloud Computing Platform,TP3

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology
© 2012 www.DissertationTopic.Net  Mobile