Dissertation > Excellent graduate degree dissertation topics show

The Research and Application of Cloud Computing Model Based on Hadoop

Author: CaoFengBing
Tutor: WuKaiGui
School: Chongqing University
Course: Computer System Architecture
Keywords: Cloud Computing Massive Data Hadoop MapReduce
CLC: TP3
Type: Master's thesis
Year: 2011
Downloads: 1042
Quote: 2
Read: Download Dissertation

Abstract

With the development of Internet technology and society, the amount and scales of network data are rapidly growing. No matter who enterprises or individuals face the problem, that is, how to store and analyze these massive data. With the growing amount of data, the traditional methods for processing massive data have many shortcomings, such as, higher cost of data storage, more difficult data management, less reliable, lower efficiency, more difficult parallel programming, and so on. A novel technology is needed to solve these problems, so the cloud computing has been brought naturally.Cloud computing integrates the free resource, provides strong storage and computes power through Internet. Hadoop is an open source distributed data processing framework. It is designed to efficiently handle massive data. Because of the features of scalability, high reliability, low cost and high efficiency, the Hadoop has become a popular development platform for cloud computing. But Hadoop has some defects, for example, the single NameNode manages the entire file system metadata and performs all the important file system operations in the Hadoop cluster, which limits the scalability and availability of the Hadoop cluster. In addition, in the traditional Hadoop, the computing and storage are tightly coupled, which also limits the performance of the cluster.The topic of this paper is the research and application of cloud computing model based on Hadoop. There are two main purposes: One is to analyze the performance bottlenecks of the tradition Hadoop framework, and to devise a new Hadoop framework; The other is, on the new Hadoop framework, to design a cloud computing model based on improved Hadoop, and to implement appropriate cloud computing platform, which focuses on solving the problem of the massive data storage and its analysis.The main works and achievements of this paper are as follows:①The basic theories of cloud computing are introduced, including the concept, characteristics and the key technology of cloud computing and some typical applications of the cloud computing platforms.②The Hadoop infrastructure is described, including the Hadoop distributed file system (HDFS) , the theory and implementation of the MapReduce parallel programming model, the optimization measures of the Hadoop framework and so on. Furthermore, the mechanisms of data management, performance guarantee and reading and writing files in the HDFS are also detailed and analyzed.③The performance bottlenecks of the Hadoop is discussed, and a new type Hadoop framework is designed to solve the two issues of the tradition Hadoop which are the bottleneck of single NameNode and the tight coupling of computing and storage. The theoretical analysis and experimental comparison demonstrate the efficiency of the new framework. The new framework has high availability and good scalability.④A cloud computing model based on improved Hadoop is designed and the main functional modules in this model are implemented by WEB technology.The newly proposed cloud computing model has many advantages, such as high efficiency, high reliability, low cost, and more convenient for users. This system model can be applied to enterprises for data storage and computation, scientific research and other fields. It provides a high reliability and low cost solution for the massive data storage and computing, and offers a novel idea for the development of cloud computing technology and the application of Hadoop.

Related Dissertations

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology
© 2012 www.DissertationTopic.Net Mobile