Dissertation > Excellent graduate degree dissertation topics show

Key Technologies Research on Personalized WEB Business Information Fusion System

Author: SuWeiBing
Tutor: YinJianWei
School: Zhejiang University
Course: Applied Computer Technology
Keywords: Information Extraction Anchor-Hop Model SNM Algorithsm Similar Duplicate Data Data Cleaning
CLC: TP393.09
Type: Master's thesis
Year: 2010
Downloads: 88
Quote: 0
Read: Download Dissertation

Abstract


As the continuous development of Internet technology, people who use the Internet technology are no longer just content with information retrieval. They need a new technology which can extract information which they are interested in from the information which is retrieved by current technology. Information fusion system is aimed to help the users with information retrieval and information extract, and becomes a research point currently with a wide range of applications. The Key technologies of WEB information fusion are information crawl, information extraction, data cleaning, information retrieval and information storage. This paper combines with the undertaken major science and technology project, focuses on information extraction and data cleaning.First, this paper introduces the background of research and subject of the information fusion module of Personal Information Push Services Project, and point the existing problems of robust and data quality in the two key technologies of information extraction and data cleaning. The problems mainly are in the low matching accuracy and low extraction efficient in information extraction and low data quality in data cleaning. At last of this section this paper introduces the main work and structures of this paper.The second part of this paper describes the key technologies, models, standards in the information fusion project. As information fusion project is aimed to process the huge information of WEB, this system uses a distributed processing framework called Hadoop, and introduces the related information of Hadoop. At last of this part, this paper describes the current research status of information extraction and data cleaning at home and abroad.The third part of this paper proposes a dynamic Anchor-Hop model based on Anchor-Hop model for the existing problems which are low efficiency and low accurate matching in the Anchor-Hop model which uses content and attributes to locate the anchor point. It is 30% faster than Anchor-Hop model in extraction experiment and the accuracy of extraction is also higher than Anchor-Hop model. In the data cleaning problem, this paper first describes the existing research on data validation, and determines the method of data validation in information fusion system, and also proposes an idea that uses the result of data validation as the feedback to the extraction system to its reliability. Second, this paper focuses on the algorithm of eliminating data duplications called SNM and field matching algorithm based on edit distance. On basis of these two algorithms this paper proposes an improved algorithm called SSNM, which first splits the key string into word segmentations, and then sorts the word segmentations and merges into a new string, then sort all the records according to the new string, and use SNM algorithm to detect data duplications. When the algorithm is computing the similarity of the two records, use the new string to compute the edit distance. The experiment results show that SSNM is better than SNM algorithm on recall rate. At last of this part, this paper gives a detailed description of the design and implementation of SSNM algorithm based on Hadoop framework.Finally this paper introduces the overall framework of information fusion system, the detailed architecture of each sub-module, and key technologies of the implementation of each sub-module.

Related Dissertations

  1. Research on Domain Entity Attribute and Event Extraction Technology,TP391.1
  2. Research on Temporal Information Recognition and Normalization,TP391.1
  3. Study on Growth Monitoring Technique Based on Pixel Un-Mixing Method and HJ Remote Sensing Images in Paddy Rice,S511
  4. Active faults based radar image information extraction method applied research and demonstration,P542.3
  5. Scholar Resume Automatic Generation Based on Text Mining,TP391.1
  6. Research and Application of Semantic-Based Information Extraction,TP393.09
  7. Reptiles theme for Education News Design and Implementation,TP391.3
  8. Web-oriented Chinese automatic summarization research generated,TP391.1
  9. Template independent web information extraction,TP393.092
  10. Study on Extraction of Coniferous Forest Information in Southern China,TP79
  11. Research on Crawling Deep Web Information,TP393.09
  12. Design and Implementation of Web Information Extraction Based on DOM,TP393.09
  13. City names addresses coding Matching,P208
  14. Study of the Scale Issues in the Information Extraction of Forest Vegetation Based on Remote Sensing,S771.8
  15. Study on Extraction of Broad-levaed Forest Information Based on Medium and High Spatial Resolution Remote Sensing Image,S771.8
  16. Crawler and Incremental Update Strategy Research in Deep Web,TP391.3
  17. Design and Implementation of EDPF NT-plus Operator Station Simulation Software,TP391.9
  18. Key Technology Research on RFID Data Cleaning,TP391.44
  19. Mobile Web Site Development Based on DOM Tree Information Extraction,TP393.092
  20. The Relation and Analysis of Relation-network Analysis Technology Based on MongoDB,TP311.13

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Computer network > General issues > The application of computer network
© 2012 www.DissertationTopic.Net  Mobile