Dissertation > Excellent graduate degree dissertation topics show

Hadoop Distributed File System (HDFS) Study Reliability and Optimization

Author: DiYongDong
Tutor: ZhouJingLi
School: Huazhong University of Science and Technology
Course: Computer System Architecture
Keywords: Distributed File System Consistency algorithm Single point of failure Hot Standby
CLC: TP316.4
Type: Master's thesis
Year: 2011
Downloads: 400
Quote: 3
Read: Download Dissertation

Abstract


As cloud computing and cloud storage gradually being accepted by the industry , more and more enterprises and research institutions are beginning to use Hadoop to develop their own cloud storage architecture system , including Yahoo!, Facebook and IBM . Because the process is mainly through Hadoop Hadoop Distributed File System (HDFS) to achieve, so the study of HDFS as many companies to structure their own cloud storage and cloud computing systems based . Therefore, for the processing of HDFS as well as its own data backup mechanisms were studied in detail . Although , HDFS itself has a very good data backup mechanism can be used to improve data security and availability . However, since there is only one metadata server nodes NameNode, which resulted in a single-point failures. By implementing Paxos consensus algorithm based on a distributed system to solve the metadata server single point of failure problem and design an electoral mechanism to improve system security and performance. In the design of the electoral mechanism , the system all the metadata server is divided into two roles Leader and Follower . Among them, a metadata server as the Leader, the other to work as a Follower . Leader election mechanism needs to be elected as a system -specific acceptor and learner to work to coordinate and synchronize all metadata servers work . There are N sets of metadata server systems, can achieve up to (N-1) / 2 sets metadata server failure , it is suitable for large-scale system. The test results, as long as the system has N / 2 1 sets the metadata server to work properly , the system can continue to work . And , Follower failure has little effect on the performance of the system , mainly in the recovery time needed for data synchronization with the Leader . The Leader failure greater impact on the system , mainly in the Leader election mechanism needs to be run after the failure to re- elect a new Leader. When the system is less than N / 2 1 survival server , the system will stop operating .

Related Dissertations

  1. The Study of the Data Fusion Technology in Multi-sensor Network,TN929.5;TP202
  2. Application of Hot-standby Embedded Controller System on the VxWorks,TP273
  3. Desgin and Study on Multi-node Hot-standby High Availability Cluster Software,TP311.5
  4. Astudy on Key Problems in Redundancy Standby Technology with High Reliability for Broadband Remote Access Server,TP309.3
  5. Weak consistency of distributed data maintenance strategy study,TP311.13
  6. In a distributed environment, Encrypting File System Design and Implementation,TP309.7
  7. Fault-tolerant software fault-tolerant computer systems design and implementation,TP302.8
  8. Research and Implementation of Ceipfs: A Distributed File System,TP309
  9. File backup system software planning and implementation,TP309.3
  10. Design and Realization of Parallel File IO Based on Hadoop Distributed File System,TP338.6
  11. The Research and Implementation of Offline Charging Collection Function in IP Multimedia Subsystem,TN915.09
  12. Design and Implementation of Distributed File System for Massive Data,TP316.4
  13. Study on Metadata Management Strategy in Distributed File System,TP316.4
  14. The Design and Implementation of a Distributed Storage and Retrieval System,TP333
  15. Design and Implementation of CE Data Receiving Station Monitoring and Task Management Software,TN927.2
  16. Research on the Application of Distributed File System Data Scheduling in G/S Model,P208
  17. The Research and Implemention of MOM’s Key Technologies,TP338.8
  18. The Research and Implementation of Server Cluster Construction Based on DFS,TP333
  19. Research on the IPDR Based Network Management System,TP393.07
  20. Design and Implementation of Fault-Tolarance Test Platform for Distributed File System,TP302.8
  21. Distributed File System Metadata Management Research and Implementation,TP338.8

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer software > Operating system > Distributed operating systems, parallel -type operating system
© 2012 www.DissertationTopic.Net  Mobile