Dissertation > Excellent graduate degree dissertation topics show

Resarch of Task-level Data Processing Based on Multicore CPU and Test of Its Performance on Cluster Platform

Author: ZhangTieJun
Tutor: ChenShuYu
School: Chongqing University
Course: Computer Software and Theory
Keywords: Multicore Cluster MapReduce Hadoop TBB
CLC: TP274
Type: Master's thesis
Year: 2011
Downloads: 60
Quote: 0
Read: Download Dissertation

Abstract


With the development of computer networks and information technology, people have to deal with increasing information growing at an alarming rate every day, especially companies like Google who provides global analysis and processing of Internet pages and it uses Google MapReduce to process P-class of web pages, audio and video files. The success of Google’s large-scale data processing promotes the development of open-source version, Hadoop MapReduce, made by the Apache community. The open-source version of MapReduce is developed using Java language. It has three components: NameNode, SecondNameNode and DataNode. NameNode is manager of the cluster with a management process JobTracker running on it.SecondNamenode is a backup server of NameNode.DataNode are a cluster of computers which do the real processing work using TaskTracker.The three components above are connected using HDFS distributed file system.When DataNode is a multicore computer and its data given are samller than 64M, the advantage of DataNode with multi-core will cease to existIt can only use one core of the computer, and the others are all in idle state. To handle this situation, this paper presents a task-level MapReduce model, use it to replace the traditional thread-level MapReduce.This paper builds a computing platform of Hadoop.after analyzing the operational mechanism of Hadoop and constraints.The corresponding solutions to subjects which prone to failure for the cluster are given.A detailed analysis of the operating mechanism of thread-level MapReduce.Then this paper points out the limitations of small blocks of data processing at thread-MapReduce level by comparision through experimentations.This article combins Hadoop MapReduce with Intel’s TBB parallel library developed using c++ programming language forming a task-level MapReduce.By using experiments of seeking PI by several groups on cluster to prove the advantage of task-level MapReduce in processing small block of data.This article also give a comprehensive comparison of these two levels of performance of MapReduce by way of experiment.and make the relationship among the number of cores , data scale and performance.

Related Dissertations

  1. Multiple Pairwise Keys Management Protocol of Function Node-Based for Wireless Sensor Networks,TP212.9
  2. Comprehensive Quality Assessment of College Students,G645.5
  3. Study on Heterosis and Genetic Basis of Soybean,S565.1
  4. The Research on Routing Protocol of Agricultural Environmental Monitoring System Based on Wir Eless Sensor Networks,TN915.04
  5. Identification of Epitope Fragments and Critical Amino Acid for Allergen TBb in Tartary Buckwheat,R284.1
  6. Body Sensor Networks Adaptive Communication Protocol,TP212.9
  7. Research on Target Location and Tracking Algorithm in Wireless Sensor Networks,TN929.5
  8. Research and Application of Map/Reduce Based Distributed Log Analyzer,TP311.52
  9. Heilongjiang Province, the problem of financing SMEs clusters,F276.3
  10. Design and Implementation of Online Shopping Prototype System Based on Hadoop,TP311.52
  11. Design of the Mobile Learning System Based on Hadoop,G434
  12. Research and Implementation of job scheduling algorithm in the Linux cluster environment,TP301.6
  13. Stainless steel industrial optimization and development mode of,F224
  14. The Research of Software Service Platform Based on Cloud Computing,TP311.52
  15. An Intrusion Detection System for High-Speed Networks,TP393.08
  16. Streaming Media Server System Based on Network Storage,TN919.8
  17. Incremental Learning Method Based on Cloud Computing,TP311.13
  18. Research and Implementation on a Distributed Service Registry Based on HADOOP Platform,TP393.09
  19. Hadoop-based video transcoding system design and implementation,TN919.81
  20. Hadoop data center deployment and tracking systems research,TP308
  21. Fault Tolerance for MapReduce in the Cloud Environment,TP302.8

CLC: > Industrial Technology > Automation technology,computer technology > Automation technology and equipment > Automation systems > Data processing, data processing system
© 2012 www.DissertationTopic.Net  Mobile