Dissertation > Excellent graduate degree dissertation topics show

A bad text filtering method

Author: ZhouJun
Tutor: YangGuoWu; XueZhenZhong
School: University of Electronic Science and Technology
Course: Software Engineering
Keywords: text weight Filting Reactionary Text text normalization non-balan ced dataset positive and negative item
CLC: TP391.1
Type: Master's thesis
Year: 2012
Downloads: 62
Quote: 0
Read: Download Dissertation

Abstract


With the rapid developing of Internet, the human enter into new era which has richinformation.But there is a new problem in this condition at the same time,in order oflawless purpose in politics or economy,many people distribute reactionary,pornographic,defrauding and violence information by use of internet,it is not good at the con-struction of statle society,value-based translation,especially the healthy growth of theyourth.How to purify the network environment and filt reactionary text is a hot researchof the construction and monitoring of the Internet now.At present,the mainstream research method of filtering reactionary texts is basedon intelligent analysis filtering of text content,mainly including the vector space modelmethod,neural networks and semantic-based filtering.The main drawback of Neuralnetworks and semantic-based filtering is complicated algorithm and slow implementa-tion, and the draoback of vector space model mainly include a large amount of docu-ments related to calculation and the lack of semantic factors, based on the above short-comings,the research direction filtering reactionary texts mainly focused on improvingaccuracy and reducing the filting time-consuming.The main purpose of this paper is todesign a method of reactionary texts filtering,while maintaining accuracy under theconditions of the maximum reduction in filtration time-consuming,when the traininglibrary is not sufficient, it is also able to achieve high performance.This article contains mainly the following contents:1、 Using a new mothod of text weight calculation.The method takes full consid-eration of the frequency of the items to be indicated in the text and the ability of distin-guish between different types of properties,which representing the text better.2、 The methods that commonly used of filtering reactionary texts is time-consuming,in this case,propose a new method of filtering reactionary texts witch can reducethe time spent on filting,and can maintain the same accuracy even better accuracy at thesame time. 3、 In the environment of Internet,training text database is always inadequate.Inthis paper,we collected two types of training text library.In this case,prove that the me-thod proposed in this paper has better filtering performance than classical methods bothtypes of training text library.4、 Based on analysing the reason for non-balanced data set and the solution of thequestion that presented by others,has present two kinds of methods that solve it.Makeuse of text normalization at the same time,it proved that the method is effecitve accord-ing to experiments.5、 Designed and validated a new filtering methods method based on the differencebetween positive and negative characteristics of items,proposed the concept of thresholdlimit value and the method for determining it.make use of normalized andnon-normalized,and get the best threshold according to experiment.6、 By analyzing the above-mentioned experiment,Get the best strategy of newmethod of filting reactionary texts.Experimental results show the effectiveness of thestrategy.

Related Dissertations

  1. Research on Basic Algorithms of Digital Image Processing and Implementation with FPGA,TP391.41
  2. Research on Facial Feature Extraction and Matching Algorithms for Image Retrieval,TP391.41
  3. Research of High Speed Image Pre-processing System Based on FPGA,TP391.41
  4. Research on Algorithms of 2D Face Template Protection,TP391.41
  5. Research of Visualization Technology in the Virtual Test of Missile,TP391.9
  6. The Research and Implemention of Image Retrieval Based on User Interested Feature,TP391.41
  7. Research of Image Mosaic Technology,TP391.41
  8. Research and Implementation of Exact String Matchiing Algorithms,TP391.41
  9. Research of Question Answering System Based on the Analysis of Lexical and Semantic Meanings,TP391.1
  10. Research on the Classification Based on the Reconstruction of Solder Joint,TP391.41
  11. Tongue Feature Extraction and Research of Fusion Classification,TP391.41
  12. Research on Structure Transition Technology for SMT,TP391.2
  13. The Fatigue State Recognition of the Driver Based on Eye Detection,TP391.41
  14. Syntactic Features Based Pronoun Resolution,TP391.1
  15. Research on Infrared Image Simulation for Aerial Objects and Background,TP391.41
  16. Design and Simulation of UHF RFID System Based on the Protocol of EPC C1G2,TP391.44
  17. Research on Intelligent Learning-Based Multi-Sensor Target Recognition and Tracking System,TP391.41
  18. Research on Image Compression and Implementation Using TMS320C6713 Based on SPIHT Algorithm,TP391.41
  19. Research on Joint Target Detection for Dual-Sensor Image and System Implementation,TP391.41
  20. Research of Images Enhancing Algorithms on Fog or Backlighting Conditions and Implementation with Hardwares,TP391.41
  21. Research of Multiple Emails Automatic Summarization,TP391.1

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Text Processing
© 2012 www.DissertationTopic.Net  Mobile