Dissertation > Excellent graduate degree dissertation topics show

Research on Web Text Segmentation Based on Conditional Random Fields

Author: CuiYanXiang
Tutor: HuangDeGen
School: Dalian University of Technology
Course: Applied Computer Technology
Keywords: Web Text Automatic Segmentation Conditional Random Fields A Latticeof Words Out-of-vocabulary
CLC: TP391.1
Type: Master's thesis
Year: 2013
Downloads: 65
Quote: 0
Read: Download Dissertation

Abstract


Natural language processing has become a hot research topic in the field of information processing. The majority of Chinese processing tasks are performed on the basis of word segmentation. After years of in-depth study, Chinese automatic word segmentation technology has achieved desirable results in terms of the traditional text. However, the segmentation of web text is still unsatisfactory, mainly due to the difference between the web text and traditional text, which proposes a new issue for segmentation.Conditional Random Fields (CRFs) don’t need the strict and independent assumptions and have overcome the label-bias problem, so it is widely applied to the Chinese automatic word segmentation, and has yielded good results. This paper, based on the character-based and word-based CRFs model, studies the method suitable for web text segmentation.The major work of this research includes four parts:(1) This research introduces Hidden Markov model, Maximum Entropy Markov model and Conditional Random Fields model and elaborates the advantages of the CRFs model in Chinese annotation.(2) This study analyzes the characteristics of the web text, improves the label set based on a character-based and word-based CRFs model, and then adopts suitable feature template for web text to improve the capabilities of our system on the web text segmentation.(3) This research proposes a method combining average mutual information and C-value to filter out the unknown word that cannot be recognized by the CRFs model. There are a lot of Out-of-vocabulary (OOV) words in the web text due to the difference between web text and traditional text.(4) This research summarizes the error of segmentation by analyzing segmentation results, and then proposes some rules-based amendments on account of the features of the web text corpus. In this way, the overall segmentation has been improved.Our experiments prove that in the test for web text corpus, our improved segmentation method based on CRFs model has given rise to an increase in the precision, recall and F-score. The feasibility of this approach is, therefore, verified.

Related Dissertations

  1. Difference in Use of English Vocabulary Learning Strategies between Male and Female Senior High School Students and the Influences on Their English Vocabulary Learning,G633.41
  2. The Effect of Learner Training of Autonomous Vocabulary Learning on Student’s English Lexical Competence in Senior High School,G633.41
  3. Research on Vocabulary Acquisition through Reading in the Perspective of Incidental Vocabulary Learning,G633.41
  4. Case Studies of Three English Teachers of Vocabulary Teaching Strategies in a Junior High School,G633.41
  5. A Frame-Based Vocabulary Teaching and Its Impact on the Senior One Students’ False Friend Phenomenon in English Writing,G633.41
  6. The Application of Mind Map in the Teaching and Learning of English Vocabulary for Senior Primary School Students,G623.31
  7. Conceptual Metaphor in high school English vocabulary teaching empirical research,G633.41
  8. High School Survey of English Vocabulary,G633.41
  9. A comparative study ( early ) foreign language textbook \,H195
  10. Teaching Strategies term \,H195
  11. Modern Chinese vocabulary words in comparison with Seoul, Korea and teaching strategies,H55
  12. The Study of Nenjiang Dialect,H17
  13. A Experimental Report on the Training of English Vocabulary Learning Strategies in Xinxiang Secondary Vocational School,G633.41
  14. An Experimental Study on the Application of Schema Theory in Teaching College English Vocabulary,H319
  15. A Comparative Study of Incidental Vocabulary Acquisition and Intentional Vocabulary Learning of Chinese English Majors,H319
  16. Foreign Language Teaching teaching color words,H195
  17. An Investigation into Vocabulary Teaching and Learning Strategies in Senior Middle School,G633.41
  18. The Effects of Word Frequency and Contextual Richness on Chinese Efl Learners’ Vocabulary Knowledge Acquisition in Reading,H319
  19. Chinese and Japanese word homograph bias analysis and teaching countermeasures,H195
  20. A Study of Category Theories in College English Vocabulary Teaching and Learning,H319
  21. A Research in Comparison of Dilao Dialect in Bobai County, Guangxi Zhuang Autonomous Region,H17

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Text Processing
© 2012 www.DissertationTopic.Net  Mobile