Dissertation > Excellent graduate degree dissertation topics show

The Study on Basic Elements to Build Statistical Language Model of Uyghur

Author: TangLiang
Tutor: WangQingXian; ShangChunGuang
School: University of Electronic Science and Technology
Course: Software Engineering
Keywords: Uyghur Language Model Perplexity Model Units Text corpus Morpheme
CLC: TP391.1
Type: Master's thesis
Year: 2013
Downloads: 12
Quote: 0
Read: Download Dissertation

Abstract


As a mathematical model to describe the inherent disciplines of natural language，language model occupy an important position in natural language processing，so it isessential to built a reliable language model in natural language processing system.Being the basic part in natural language processing of Uyghur，Uygur language model iswidely used in the field of Speech recognition, machine translation, informationretrieval, etc. However, at the present time, the study of Uygur language model is just atthe beginning stage，so a further study of this model will be of great significance for theinformation-based development of Xinjiang region.This dissertation aimed to establish Uyghur language model according to differentmodel units. It means to find the best smoothing method and units to build Uyghurlanguage model. The contents of this dissertation are as follows:To solve the problem of data sparseness，many smoothing algorithms such asAddition smoothing, Good-Turing smoothing, Witten-Bell smoothing, Katzsmoothing, absolute discount smoothing, Kneser-Ney smoothing were studied. Theexperimental results show that the perplexity of absolute discount smoothing was best.The experimental data were collected based on Uygur spoken dialog corpora ofphone，and text corpora from bilingual teaching system and some daily expression.After pretreatment, these data were processed into Uygur text corpora. Two wordsegmentation methods were adopted，one was Uyghur words segmentation methodbased on dictionary and the other was segmented in the unsupervised form.Based on Uyghur segmentation, the traditional N-gram statistical language modelwas improved. The Uyghur words can be divided into different units, using these units，three kinds of Uyghur language model were built and N-gram Language model basedon morphemes class was proposed. In this thesis，a series of experiment were conductedusing SRILM1.5.12toolkit and MITLM0.4toolkit，the results showed that theperplexity of the Uyghur language model based on morphemes was far below that basedon word. And the confusion degree of the former was reduced to about2/3of the latterone.

Related Dissertations

  1. Research on Structure Transition Technology for SMT,TP391.2
  2. Word Sense Disambiguation Corpus Automatic Acquisition,TP391.1
  3. A Research on the Correlation between Uyghur Classification of Hilit and Immune Globulin , Complement, Serum C-peptide, Glycosylated Serum Protein,R29
  4. Comparisons of Taste Words in Uyghur, Chinese and English,H136;H313
  5. Ink Morpheme in the Application of Poster Design,J524
  6. The Research and Implement of Chinese Input System Based on Dynamic Self-adaptation Language Model,TP391.14
  7. A Design and Practice Research of Bilingual Teaching of Junior High School Geography in Xinjiang,G633.55
  8. Traditional silk craft Uighur Aidelaisi,J523.1
  9. An Inquiry into Uyghur Feuilletons,I207.9
  10. Uyghur text study and text converters Design and Implementation,TP391.1
  11. The Cultural Research on the Design of Xinjiang Uyghur Carpet,J523.3
  12. Study on Proses in Qochu Uyghur Literature,I207.9
  13. "I Abudurexiti Tiku Er Poems by Wood " Translation of Aesthetics,I207.25
  14. Analysis on 80’s Generation’s Poetry on Uyghur Literature,I207.25
  15. Numerals in Chinese and Uyghurculture Meaning and Translation,H215
  16. Research on Linked Form in Uyghur Discourse,H215
  17. In the Process of Extra-linguistic Factors and Translation Between Chinese and Uyghur Language,H215
  18. Research on Pair Words in Maitri Simit,H211.4
  19. The Euphemism of Uyghur and Its Chinese Translation,H215
  20. Study on Translating Tactics and Culture Differences between Chinese and Uyghur,H215
  21. A Study on Cultural Image of Chinese Poetry Conveyed in the Transformation of Uyghur,H215

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Text Processing
© 2012 www.DissertationTopic.Net  Mobile