Dissertation > Excellent graduate degree dissertation topics show

Study on Disambiguation Algorithm for Chinese Word Segmentation

Author: LiuYuZi
Tutor: HeZhongShi
School: Chongqing University
Course: Computer Software and Theory
Keywords: Natural Language Processing Word Segmentation Crossing ambiguity Combinatorial ambiguity Hidden Markov Model Part of Speech Tagging Support Vector Machine
CLC: TP391.1
Type: Master's thesis
Year: 2005
Downloads: 375
Quote: 16
Read: Download Dissertation

Abstract


NLP (Natural Language Processing) is not only an important direction in the field of computer science but also an important branch of Artificial Intelligence. Chinese word segmentation is the foundation of NLP. There are many reasons that operated against the development of the Chinese Word Segmentation. Solving the ambiguity problem can be considered as the most important reason of these. Aiming at ambiguity in Chinese word segmentation, this paper studies the merits and disadvantages of solving ambiguity. It is found that the previous approaches can’t satisfy the practical application. Two kinds of algorithms to solve ambiguity are proposed in this paper, which based on these researches. The two algorithms proposed in this paper form an independent modularity, which are to be applied in “Automatic Chinese Word Segmentation System”. This paper introduces chiefly the NLP at first. Then it advance there is no natural segmentation mark in Chinese words, which is one of the biggest differences between Chinese and English in NLP researches. So it should be an automatic segmentation process before dealing with the input text to perform further researches. With the development of automatic segmentation, more and more researchers have focused on solving the ambiguity problem. We study algorithm of solving ambiguity at the main part of the paper. Firstly ambiguity’recognition methods are introduced. Then two algorithms are presented after analyzing the insufficiencies and advantages of these algorithms. One is an algorithm, denoted by HB, which is based on Hidden Markov Models and word Bigram to deal with the crossing ambiguity. The key idea of the HB algorithm is to combine word Bigram with POS Bigram. This algorithm deals with the part of speech and provides a new way for solving the crossing ambiguous. Another one is an algorithm, denoted by SR, which is based on the Support Vector Machines (SVM) and the rules to deal with the combinatorial ambiguity. The key idea of the SR algorithm is to solve combinatorial ambiguity making use of the theory of SVM and rules of parts of speech. At the ending part of this paper,these two algorithms are applied in “Automatic Chinese Word Segmentation System”, which was developed by our team. In open and closed test of various types of Chinese corpus, we made a comparison between the two algorithms and ICTCLAS System, which is developed by the Chinese Academy of Sciences. The experimental result shows that the accuracy of segmentation for ambiguity has been improved greatly. It is also proved that the two algorithms are feasible to solve ambiguity. Then the paper makes a conclusion and gives suggestion for the future researches.

Related Dissertations

  1. Research on Automatic Detection Algorithm for Substructure Distress of Highway Pavement Based on SVM,U418.6
  2. Research on Autamatic Music Structrue Analysis,TN912.3
  3. Packet Loss Recovering Technology for Speech Transmission over Network,TN912.3
  4. Research on Domain Entity Attribute and Event Extraction Technology,TP391.1
  5. Word Sense Disambiguation Corpus Automatic Acquisition,TP391.1
  6. Research on Transductive Support Vector Machine and Its Application in Image Retrieval,TP391.41
  7. Fault Diagnosis Method Based on Support Vector Machine,TP18
  8. Research for Infrared Image Target Identification and Tracking Technology,TP391.41
  9. Prediction of Binding Affinity of Human Transporter Associated with Antigen Processing,R392.1
  10. Research on Semantic Role Labeling for Chinese Nominal Predicates,TP391.1
  11. The Forecast Methods of Mining Risk of the Coal Resources Affected by Dynamic Disaster,TD713
  12. Research on Application to Video Scene Analysis of Hidden Markov Model,TP391.41
  13. Research of Coal and Gas Outburst Prediction Based on Data Mining Technology,TP311.13
  14. Voice recognition in the access control application,TN912.34
  15. Chinese word segmentation based on understanding of system design and implementation,TP391.1
  16. The Research on Full-Text Search and Related Technologies,TP391.3
  17. Research and Implement of Chinese Word Segment Techniques Based on the Conditional Random Field,TP391.1
  18. Research and Realization of License Plate Character Recognition Algorithm Based on SVM,TP391.41
  19. Research on Approaches of the Subjective Automated Assessment,TP391.1
  20. A Study of Marine Oil-spill Identification System Based on Information Fusion,X55
  21. Study of Structuralization for Electrocardiography Diagnostic Report,R444

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Text Processing
© 2012 www.DissertationTopic.Net  Mobile