Dissertation > Excellent graduate degree dissertation topics show

The Applicability of Zipf’s Law in Chinese Language Based on Words’ Frequency Statistics

Author: HeFengYuan
Tutor: WangXiuLi
School: Anhui University
Course: Chinese Philology
Keywords: word frequency statistics Zipf’s first law Zipf’s second law the applicability of Chinese language
CLC: TP391.1
Type: Master's thesis
Year: 2011
Downloads: 65
Quote: 0
Read: Download Dissertation

Abstract


In this paper the main problems to be solved by large Chinese text corpus of word frequency statistics and analysis, including Zipfs first law and Zipfs second law, including the applicability of the validation studies conducted in Chinese.To carry out this research work is divided into five chapters:The first chapter is a general introduction and word frequency statistics for an overview and clarify the definitions and characteristics of word frequency, word frequency statistics of foreign and domestic described the development process of thesis describes the purpose, significance and content.The second chapter is the subject of this study-the development process of Zipfs law studies, this paper explains the theoretical background and philosophy, mathematics on the alignment from the husband’s law is deduced and presentation, and review for the domestic Zipf’s law and the applicability of Zipf’s law studies in Chinese.The third chapter is a large-scale text corpus by word frequency statistics and analysis to verify the first law of Zipf’s Chinese applicability. This article first defines the word level and word order to distinguish clear the way for the next experimental obstacles; followed by experiment 1 on the word-level verification and comparison of methods for determining, selecting a more appropriate method for determining the word class; then experiment 2 on a language test materials and statistical manual segmentation, segmentation means to compare the computer and artificial means of statistical difference between the computer means to verify the feasibility and credibility; Finally, experiment 3, large-scale text corpus word frequency statistics and analysis Corpus six points plotted Zipf’s distribution curve and distribution curve of the number, and with the first law in the Zipf’s plot of the Zipf’s distribution curve ideal and the ideal Zipf’s distribution curve compared to the number to determine Zipf’s first law of Chinese applicability.The fourth chapter is the second law of husband aligned text corpus for large-scale verification and analysis to determine the distribution of low-frequency words the law section of the Chinese word frequency, and the alignment of the applicability of the second law husband.Zipf paper described the development process of the second law, and Zipfs first law with distinction and connection; then set the experiment 4, the first five minutes corpus corpus statistics of the same frequency and number of words number of words with frequency logarithm, then the second law of Zipf’s, the number of words with the theoretical frequency derived to calculate the predicted number of words with the same frequency and the predictive value of the number of same frequency on the number of words, and finally proposes to draw five word corpus of their respective distributions of the same frequency curve, with the number on the number of word-frequency distribution curve, the number of predicted words with the same frequency distribution curve and predictive value of the log number of words with the same frequency distribution curve, for comparison, in order to determine the second law of Zipf’s Chinese applicability.The fifth chapter is the conclusion, made on this article summarizes statistics and verification, to be reflection on the shortcomings of the future can continue to work for Outlook.Corpus of six points in the Zipf’s distribution, we found that large Chinese text corpus word frequency distribution of words in the high-frequency words and IF stages of the first law consistent with Zipf’s distribution, while the low-frequency words while the Frequency section and the zipf’s distribution of the second law husband is more consistent.Accordingly, the large Chinese text corpus term frequency distribution of low-frequency words than paragraph described the Zipf’s first law of the linear descending trend of significant deviation, showing a parabolic decline in the state; in which the high-frequency words of paragraph Zipf’s word frequency distribution and the second law of distribution is not described in similar ladder-like decline.It also reflects Zipfs first law and second law of their scope and range.This paper concludes the large-scale Chinese text corpus distribution consistent with Zipf’s law.

Related Dissertations

  1. Study on Mere Exposure Effect Using Chinese Character Frequency Statistics,B842
  2. A Study of How to Judge Plagiarism in the Academic Theses Based on the Technology of Text Mining,TP391.1
  3. Research on Search Technology of Chinese Information,TP391.3
  4. A Comparative Study of Words in English and Chinese Advertising,H315
  5. Chinese Word Auto-segmentation Design and Algorithm Realization for Chinese Network Information Retrieval,TP391.3
  6. Corpus-based Dongtai dialect characteristic word definitions,H172.4
  7. The Experimental Study on the Corpus Based Word Frequency Statistics of Modern Uighur Language,H215
  8. Research on Basic Algorithms of Digital Image Processing and Implementation with FPGA,TP391.41
  9. Research on Facial Feature Extraction and Matching Algorithms for Image Retrieval,TP391.41
  10. Research of High Speed Image Pre-processing System Based on FPGA,TP391.41
  11. Research on Algorithms of 2D Face Template Protection,TP391.41
  12. Research of Visualization Technology in the Virtual Test of Missile,TP391.9
  13. The Research and Implemention of Image Retrieval Based on User Interested Feature,TP391.41
  14. Research of Image Mosaic Technology,TP391.41
  15. Research and Implementation of Exact String Matchiing Algorithms,TP391.41
  16. Research of Question Answering System Based on the Analysis of Lexical and Semantic Meanings,TP391.1
  17. Research on the Classification Based on the Reconstruction of Solder Joint,TP391.41
  18. Tongue Feature Extraction and Research of Fusion Classification,TP391.41
  19. Research on Structure Transition Technology for SMT,TP391.2
  20. The Fatigue State Recognition of the Driver Based on Eye Detection,TP391.41
  21. Syntactic Features Based Pronoun Resolution,TP391.1

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Text Processing
© 2012 www.DissertationTopic.Net  Mobile