Dissertation > Excellent graduate degree dissertation topics show

Mining Bilingual Parallel Corpora from Web Automatically and Its Application in Statistical Machine Translation

Author: LinZheng
Tutor: MaXiRong
School: Tianjin Normal University
Course: Applied Computer Technology
Keywords: Web Mining Parallel Corpus Sentence Alignment Statistical machine translation
CLC: TP391.2
Type: Master's thesis
Year: 2010
Downloads: 140
Quote: 0
Read: Download Dissertation

Abstract


The Parallel Corpus There are many important applications in the field of natural language processing, it provides the essential training data for statistical machine translation, is also an important basis for resource dictionary compilation and cross-language information retrieval applications. However, large-scale bilingual parallel corpus is not easy to obtain, existing parallel corpus in scale, timeliness and balance of the field can not meet the actual need to deal with real texts. With the popularity of the Internet and the rapid development, more and more bilingual website was created, more and more information is released in the form of multi-language, which provides a great source for the construction of bilingual and multilingual corpus. Some researchers have proposed the automatic mining method proposed effective solution for bilingual or multilingual parallel corpus automatically build a Web-based bilingual or multilingual parallel corpus. This article is committed to build a large-scale Web-based bilingual parallel corpus automatic acquisition system. To obtain the main results of the following aspects: 1. Bilingual mixed pages automatically discover and get the bilingual parallel resources on the Internet is divided into two categories: one is the bilingual resource distribution between the two pages, two pages with different languages describe the content on the translation, which we call the bilingual parallel pages; other bilingual resources are located within the same Web page, which we call the bilingual mixed pages. Previous system web-based bilingual parallel, but through observation, we found that there are a lot of bilingual mixed pages on the Web, and the bilingual resource control in the bilingual mixed pages neat, higher translation quality is very valuable bilingual resources sources. Bilingual parallel web address or structure similarity, the approach has been very mature, but these methods are not applicable to bilingual mixed page. The candidate bilingual mixed web distribution is often uncertain, and the lack of some common heuristic information, and more difficult to obtain. This paper presents a strategy based attempts to download automatically discovers bilingual mixed page, use the method to get candidate mixed Website has a higher rate of correct. 2. Extracting parallel sentence pairs from a bilingual mixed pages extracted from the bilingual mixed pages in parallel sentence pairs of the main tasks can be divided into three parts: web noise filtering, the bilingual mixed pages confirm and sentence alignment. Of this study, and two pages to noise method: the dedicated template-based method and general method based Html tags tree. A two-step experiment, bilingual mixed pages confirm this article are based on crude discriminant and bilingual character dictionary-based fine discrimination. Finally, this paper uses bilingual parallel sentence pairs into chapter bilingual parallel text sentence alignment method based on mixed information. In this paper, these three difficult problems to solve, a a bilingual hybrid web-based parallel corpus mining system automatically. Study Web bilingual parallel corpus in the practical application of this article from the Web to obtain bilingual parallel sentence applied to statistical machine translation model training sentence different sort of quality and information retrieval in the field of two kinds of application strategy the Web corpus load to the training set, the experiments show that the proposed two strategies can improve the performance of the translation system in the the IWSLT evaluation task in the BLEU value can be increased by 2-5 percentage points.

Related Dissertations

  1. Research and Implementation of Mining Implicit User Interest,TP311.13
  2. The Research of Decoding Algorithm for Statistical Machine Tranlation,TP391.2
  3. Web-based Mining Technology and Its Application in Digital Library,G250.76
  4. The Research and Application with Web Mining in E-Commerce Recommendation System,TP391.3
  5. Advanced alignment machine translation technology and development set selection strategy study,TP391.2
  6. A Corpus-Based Study on the Explicitation of Interpersonal Meaning of Lord in the Chinese Translations of Shakespeare’s Plays,H315.9
  7. The Research on Public Opinion Mining and Group Behavior Analysis on the Internet,F49
  8. Research and Implementation on Topic Search Oriented to Enterprise Competitive Intelligence,TP391.3
  9. The Study on the Personalized Teaching Platform under E-learning Environment Based on Web Mining,TP391.6
  10. Research on Intelligent Recommendation Based on Web Data Mining,TP391.3
  11. Research on Reordering Problems of Hierarchical Phrase-Based Translation Model,TP391.2
  12. Sentence- aligned parallel corpus and its application in machine translation,H315.9
  13. A Corpus-based Study on Lexical Chunks in College English Writing,H319
  14. Research on Personalized Recommender System Based on Collaborative Filtering Algorithm,TP393.09
  15. Research on Blog Friends Recommendation Mechanism in Blogsphere,TP393.092
  16. Study on Collaborative Filtering Recommendation Based on Users’ Interest Clustering,TP393.09
  17. Research on Community Structure of Complex Network Based Website Clustering,TP393.092
  18. Cross-language Text Classification Research Based on Latent Semantic Dual Space,TP391.1
  19. Research on Web User Access Clustering Pattern,TP311.13
  20. The Study and Implementation of Topic Search and Web Mining,TP391.3
  21. Research and Implementation About Dynamic Balance of Computing Resources in Online Web Mining Base on Cloud Platform,TP311.13

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Translator
© 2012 www.DissertationTopic.Net  Mobile