Dissertation > Excellent graduate degree dissertation topics show

Study on Metadata Management Strategy in Distributed File System

Author: LiXin
Tutor: PeiXiaoBing
School: Huazhong University of Science and Technology
Course: Software Engineering
Keywords: Distributed file system Metadata management strategy System scalability Label partitioning Directory index
CLC: TP316.4
Type: Master's thesis
Year: 2010
Downloads: 150
Quote: 0
Read: Download Dissertation

Abstract


In recent years, with rapid increment of data in all kind of fields, distributed file system is facing with the big performance challenge caused by many more files and data to be stored. As the most important part of distributed file system, metadata management system is a critical aspect of overall system performance. However, limited by traditional technologies of metadata partitioning, existing metadata management system is inefficient for the problem. A new metadata management strategy is proposed, whose partition granularity is subdirectory naming label. In the new strategy, the partition granularity is label. The granularity of metadata partition has an important impact on many aspects of metadata processing, such as concurrency controlling, cache utilization, load balancing and system scalability. Based on the analysis on traditional partition methods, label is thought to be better.In addation, a large directory is organized by extendible hashing. In the new metadata management strategy, a directory is divided into multiple labels and all labels are distributed among metadata servers. Meanwhile, each label is also divided into multiple chunks that are responsible for containing files.An index server is separated from metadata servers. The directory attribute metadata is accessed most frequently among all kinds of metadata, so managing the directory metadata separately by an index server is a good method to reduce the overload of a metadata server.Load balancing is implemented by copying popular metadata temperately. When some metadata get popular in a metadata server, these metadata will be copied and migrated to another metadata server, which will distribute accesses to these popular metadata among servers caching them; Meanwhile, system scalability is guaranteed by consistent hashing. Whenever a metadata server is added or removed from server cluster, ordinarily only k/n metadata need migrate among servers, where k is the number of metadata and n is the number of metadata servers.At last, a metadata management system is implemented on the basis of the new metadata management strategy, and experiments prove that the new strategy is better than subtree partitioning strategy.

Related Dissertations

  1. Weak consistency of distributed data maintenance strategy study,TP311.13
  2. In a distributed environment, Encrypting File System Design and Implementation,TP309.7
  3. Hadoop Distributed File System (HDFS) Study Reliability and Optimization,TP316.4
  4. Distributed File System centralized security management server design and implementation,TP316.4
  5. Research and Implementation of Ceipfs: A Distributed File System,TP309
  6. The Research and Development on Supplementary Scheme System,TP311.52
  7. Design and Realization of Parallel File IO Based on Hadoop Distributed File System,TP338.6
  8. Design and Implementation of Distributed File System for Massive Data,TP316.4
  9. The Design and Implementation of a Distributed Storage and Retrieval System,TP333
  10. Research on the Application of Distributed File System Data Scheduling in G/S Model,P208
  11. The Research and Implementation of Server Cluster Construction Based on DFS,TP333
  12. Design and Implementation of Fault-Tolarance Test Platform for Distributed File System,TP302.8
  13. Distributed File System Metadata Management Research and Implementation,TP338.8
  14. A Similar Image Search Engine Based on Millions of Images and Distributed Computing,TP391.41
  15. For multi- tasking, multi- channel parallel crawler technology research,TP391.3
  16. Research of Distributed File System Dedicated to Massive E-Mails’Storage,TP393.098
  17. Design and Implementation of a Backup System Based on Data De-Duplication,TP309.3
  18. The Research and Improvement for General Distributed File System,TP316.4
  19. Research and Design of the Software Architecture of the Intelligent Network Storage System’s Server,TP333
  20. Research on the Application of Distributed File System in G/S Model,TP316.4

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer software > Operating system > Distributed operating systems, parallel -type operating system
© 2012 www.DissertationTopic.Net  Mobile