Dissertation > Excellent graduate degree dissertation topics show

A Distributed Graph Storage and Query System for Web Data Management

Author: TaoDao
Tutor: ZhouAoYing
School: Fudan University
Course: Computer Software and Theory
Keywords: TLGM data model Distributed Storage Web Data Management
CLC: TP393.07
Type: Master's thesis
Year: 2009
Downloads: 194
Quote: 0
Read: Download Dissertation

Abstract

With the rapid development of the World Wide Web (the World Wide Web, WEB or WWW) size and application explosive growth of Web data, Web data has the largest database in the world. Web data related data such as search records variety of Web services using the recorded data and so on in rapid growth. Compared with the traditional data, Web data has unstructured, fast growth, and the characteristics of a variety of data types, which makes the processing of Web data and the existing data processing mode there is a large difference. Web data processing technology in all areas of reality, has a wide range of needs and applications, Web data has become a focus of computer science today. In practical applications, the CWI, need a large number of Web data stored and indexed, and based on this query keyword and structural information. TLGM [2] and TLGM-QL, [3] as part of the CWI meet the above requirements. This paper mainly focuses on storage of TLGM data model in a distributed environment, and to achieve the four basic operator TLGM-QL. In the design and implementation process, we found that the unbalanced nature of the real environmental data will lead to the degradation of the storage and query algorithms, thus reducing efficiency. To solve these problems, this paper proposes a series of balanced measures to ensure that each node's computing and storage load differences remain within a reasonable range. On this basis, we propose a a new subgraph reconstruction algorithm, in order to support the query graph structure and balanced measures to ensure the efficiency of the algorithm. In this paper, experiments of virtual data and real environmental data to verify the effectiveness of the system. The contribution of this article and innovations are summarized as follows: This paper analyzes the characteristics of Web data, and introduced TLGM model to illustrate the difference between the chart data and traditional data storage, indexing and query. Firstly, in a centralized environment to analyze the possibility of using a relational database to store map data, collected a number of map data, and designed a series of queries through experiments to verify the condition of data storage and query efficiency, It showed the deficiencies and limitations of centralized storage. 2. Analysis of the the characteristics of TLGM, the realization of the data model to store and query methods elaborated in a distributed environment. On this basis, we summed TLGM Figure data model to meet the query conditions, based on the four basic operators, and illustrate the operator has good scalability. At the same time we are given in a distributed environment, the realization of the operator method and the specific algorithm. Proposed new diagram reconstruction algorithm, describes how to use the algorithm to achieve query graph structure. By MapReduce [4] framework to implement this algorithm, the algorithm has good scalability, and allow the results to improve the efficiency of our reconfigurable cache. In the realization of the process, we found that the different data nodes the load quite different, so as to achieve load balancing by certain modifications of the original algorithm. The same time, we generate and collect certain graph data, and through a series of experiments to verify the effectiveness of the method. In summary, we collate and analyze the problem of Web data storage, indexing and query and through TLGM model into a map data storage, indexing and query this. According to the experimental results, we have identified to MapReduce distributed framework as a basis, designed and implemented on top of this diagram data of the four basic operators and Fig reconstruction algorithm, experiments show that the results of our study have good efficiency and scalability.

Related Dissertations

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Computer network > General issues > Computer Network Management
© 2012 www.DissertationTopic.Net Mobile