Dissertation > Excellent graduate degree dissertation topics show

Pattern-Aware Data Reorganization in MPI-IO

Author: HeJun
Tutor: XuCheng
School: Hunan University
Course: Computer System Architecture
Keywords: data reorganization MPI-IO PVFS2 parallel I/O parallel computing
CLC: TP316.4
Type: Master's thesis
Year: 2012
Downloads: 3
Quote: 0
Read: Download Dissertation

Abstract


In the past decades, high performance computer has made scientific computing, modeling and simulation in many domains at large scales faster and more efficiently, which is a great opportunity for finding the solutions of challenging problems. Lots of scientific applications are data-intensive. For example, colliders, space telescopes and nuclear simulations generate tons of data per second, which requires high performance I/O. However, when the CPU speed has been growing following the Moore’s Law, the speed of I/O has not been growing accordingly. The gap between CPU and I/O makes I/O become the bottleneck of the system, which is the so called "I/O Wall" problem, which needs to be immediately addressed.MPI-IO and parallel file systems are widely used as ways to mitigate the I/O bottleneck. They bridge the gap between CPU and I/O speed by improving the parallelism of data accessing. In parallel file systems, the sizes and contiguousness of the data requests are two of the most important factors when it comes to performance. However, application developers organize the data by their logical understandings of the data, which may bring to the parallel file systems lots of small non-contiguous I/O requests and seriously degrade the performance. In this article, we propose an approach to match data access patterns and features of parallel file systems by reorganizing the data and eventually improve the performance of I/O systems. The major work we have done is as follows:1. We propose pattern-aware data reorganization in MPI-IO in order to enhance the contiguousness and to reduce the number of requests of data accesses. The major advantage of reorganizing the data in MPI-IO is that the layers below MPI-IO can use the improved data access patterns and then further enhance the performance. In the proposed approach, we analyze the data access traces and reorganize the data based on the access patterns we are aware of. The future data accesses have better patterns which can use the file systems efficiently.2. Based on the strategies stated above, we design a pattern-aware data reorganization system. First, we build a remapping table from the analysis of data access traces. When the application is run again, the remapping table will be loaded to the memory. Upon the arrivals of new requests, our system checks if they match the patterns already built in the remapping table. If they do, the remapping layer in MPI-IO will convert the old requests to new request for the reorganized data.3. We implemented and tested pattern-aware data reorganization system in MPI-IO and PVFS2(Parallel Virtual File System2). In this article, we compare the I/O-signature-based remapping table with a traditional remapping table. The results show that our I/O signature-based table takes less time for searching and less space. In addition, we tested and confirmed that the pattern-aware file has good fault-tolerance capabilities. Moreover, we tested and analyzed the performance by IOR and MPI-TILE-IO. By placing data in a way favoring the parallel I/O system, gains of up to two orders of magnitudes in reading and up to one order of magnitude in writing were observed with spinning disks and solid-state disks.

Related Dissertations

  1. Research on Parallel Frequent Graph Pattern Mining,TP311.13
  2. Task Partition of Network Simulation under Large-Scale Computing,TP393.01
  3. Analysis and Control of 6-DOF Electrical Driven Parallel Manipulator,TP242.2
  4. Designs and Applications of Fuzzy Synthetic Evaluation Models Based on Parallel Algorithms,TP18
  5. Study on Small Bank-based Constructed Wetland for Remediation of Polluted Water in City Stream,X703
  6. Research on the Online Parallel Connection Examination and Approbation System,D630
  7. Visual Feedback and Memory Behavior Based GPU Parallel Ant Colony Algorithm,TP301.6
  8. Functional Evaluation of Conparat Platform for Concordancing Parallel Texts,H08
  9. The Airborne LiDAR and LiDAR Points-cloud’s Quick Processing Method,TN959.73
  10. Research on Noise Tolerance of Morphological Associative Memory Networks,TN911.4
  11. Research on Image Digital Watermarking Technology Based on CUDA,TP309.7
  12. Study of 3D Stratum Modelling and Visualization Method Based on MPI,TP391.41
  13. Design and Implementation of Image Authentication on CUDA Platform,TP391.41
  14. The Research of Orthophoto Generation Based on GPU Parallel Acceleration,TP391.41
  15. Research on Sentence Structure of English and Tibetan in a Bilingual Parallel Corpus,H214
  16. The Designing of HD Vehicle Detection System Based on Fpga,TP391.41
  17. The Kinematics and Dynamics on the Cross-rod Parallel Machine Tools,TH113
  18. A Corpus-Based Study on English Personal Pronouns in Political Speeches and Statements,H314
  19. The Research on Feature Selection for Data Stream,TP311.13
  20. Study on the Performance of Mixing in the Jep Loop Reactor,TQ052
  21. A Thesis on Rights Restriction of Trademark,D923.43

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer software > Operating system > Distributed operating systems, parallel -type operating system
© 2012 www.DissertationTopic.Net  Mobile