Dissertation > Excellent graduate degree dissertation topics show

A Back-off Smoothing Algorithm of Language Model Based on Mutual Information

Author: ZuoKun
Tutor: ZhangLei
School: Harbin Engineering University
Course: Signal and Information Processing
Keywords: statistical language model sparse data smoothing technique mutual information
CLC: TP391.1
Type: Master's thesis
Year: 2009
Downloads: 38
Quote: 0
Read: Download Dissertation

Abstract


Natural Language Process is an attracting and challenging field in computer science. Its purpose is to establish a computing model by which can simulate man’s language cognizing processes. Whereas, the intelligence of today’s computer is far behind the man’s and cannot being mentioned in the same level with it. The factors obsessing its development are various. Data Sparseness in Statistical Language Model (SLM for shortly)is one of the problems needs being solved in the Natural Language Process field. This paper aims at the widely popularized and used Statistical Language Model, researches the existing ways of establishing models and the Smoothing Techniques, and brings forward a new way to establish a model which can suffice in unitary feature of probability and a new smoothing technique based on Mutual Information. This smoothing technique combines the ideas of mutual information (MI for shortly) and entropy, imposes the theory of non-linearity system optimization. The main outputs of this paper are listed below:First of all, The paper firstly introduces the theory of probability and information about the knowledge of statistical language model, then bring in the smoothing techniques of statistical language model exists detailed.After analyzing the establishing of the unigram、bigram and trigram model, The main body of this paper brings a way of establishing the SLM which can suffice in unitary feature of probability. Then give some necessary processes about training texts before establishing. This paper also introduce some language toolkit pages, and introduce the process of establish the SLM by some results of using the CMU_Cambirdge_Tooklit.Furthermore, this paper brings forward a new smoothing technique based on Mutual Information. This algorithm not only analyzes the coupling relation between words in bigram, but also adjusts the unreasonable frequency distribution of events appeared. The probabilities of bigram are discounted differently according to the mutual information. For unseen events, the probabilities are back-off to low-order model. And it gains the coefficient of the smoothing formula based on the theory of non-linearity system optimization by minimizing the perplexity of the model, so as to ensure the superiority of the method. This paper also analyzes the inner regulations of language, and redefines the formula of Mutual Information which includes the factor of the bigram’s frequency. With this new definition, the algorithm in this paper is more reasonable.At the end part of the paper, it compares the new smoothing technique with the existing smoothing techniques. Test the perplexity of model in testing corpus of different domains, all the perplexities of the proposed smoothing algorithm descend more than 20% compared with the traditional Katz algorithm. The rationality of redefining the MI formula is proved by the results of experiments. The conclusion for this paper comes to the end of the paper.

Related Dissertations

  1. Word Sense Disambiguation Corpus Automatic Acquisition,TP391.1
  2. Study of Medical Image Registration Methods Based on Multi-features and Artificial Immune Algorithm,TP391.41
  3. Node Importance Evaluation and Its Application Research in Complex Networks,O157.5
  4. The Research of the Medical Image Registration Based on New Criterion with Improved Optimal Strategy,TP391.41
  5. Medical Image Registration Based on Mutual Information and Wavelet Transformation,TP391.41
  6. Kullback-Leibler distance framework abstracts retrieved in Retrieval,TP391.3
  7. Research and Implementation of Convertion for Mongolian Presentation Character to Basic Letter,TP391.1
  8. Chinese-Mongolian Statistical Machine Translation System Based on Affix Features,TP391.2
  9. Researches on Multi-Model Medical Image Registrtion Technique,TP391.41
  10. The Research about Influencing Factors of Using Estimation Formula to Estimate Variance Compoents under Sparse Data Matrix,O212.1
  11. Research on Image Registration Method Based on Empirical Mode Decomposition,TP391.41
  12. Research of Signal Separation in MIMO Multimode Fiber Communication Based on ICA,TN929.1
  13. With Edge Information multispectral images mutual information algorithm,TP391.41
  14. The Application Research on Interactive Theory in Constructing Information Learning Environment,G434
  15. Recognition of Chinese Prase,TP391.1
  16. Research and Analysis of Brain Function Based Based on Complex Network,R318.0
  17. First Arrival Pickup of Seismic Waves Based on Mutual Information,P631.4
  18. Research on Feature Selection for Pattern Classification,TP391.41
  19. Cognitive radar waveform design algorithm,TN958
  20. Study of Image Registration Theory and Algorithm,TP391.41
  21. Study on Real-time Dynamic Range of CCD Imaging Based on Reflective Liquid Crystal,TP391.41

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Text Processing
© 2012 www.DissertationTopic.Net  Mobile