Dissertation > Excellent graduate degree dissertation topics show

Content-based mining Chinese spam filtering technology research

Author: XuLiPing
Tutor: MaGang
School: Dongbei University of Finance
Course: E-commerce
Keywords: Spam Bayesian Feature Selection Mutual Information Weka
CLC: TP393.098
Type: Master's thesis
Year: 2010
Downloads: 68
Quote: 0
Read: Download Dissertation

Abstract


As computer networks and the rapid development of communication technology, e-mail has become an important means of modern communication. However, people enjoy the convenient e-mail service, but also suffered the annoyance of spam. Growing proliferation of spam, not only a waste of network resources, violation of the user's personal interests, or even destroy social security and stability, and therefore more and more public attention and concern of researchers, spam filtering technology has become a hot research world. Firstly, in-depth study of a large number of domestic and international anti-spam literature and data, systematic analysis of the problem of spam in the background and current situation; Then, further information is closely related with the e-mail and e-mail SMTP protocol works, analyzes the E-mail spam security flaws and reasons for the proliferation; Finally, in the e-mail works on the basis of research, in-depth analysis of the content-based spam filtering technologies, including e-mail text extraction, Chinese word processing, feature value selection, Text representation of these parts. For spammers, designed in order to avoid interference filter information before processing the segmentation carried out to extract plain text message preprocessing and feature words ahead of pre-dimensionality reduction, which effectively reduces the dimension of feature words , greatly improves the efficiency of the algorithm; exist due to some common vocabulary and rare, rare occurrences of words on the classification of the contribution is small, therefore, this paper segmentation process, increasing the removal of stop words and sparse words this process, and gives a new segmentation algorithm processes; in the use of mutual information feature selection algorithm in the process of mutual information algorithm for spam filtering shortcomings, from the frequency, concentration and negatively related to three aspects the traditional mutual information algorithm proposed to improve the algorithm by adding the feature words in a document to show the frequency of this factor. Finally, this paper proposes the use of \subset. To verify the improved Chinese word segmentation effect, the use of Wuhan University of ROSTContentMining software, respectively before and after improvement of the segmentation algorithm for Chinese word processing; in segmentation, based on a real set of messages in MATLAB using mutual information algorithm for feature selection of words, the results show that this improved algorithm for mutual information, the \information value can play a greater role on the classification category; Finally, the classification in the mail section, select 10-fold cross validation method in Weka platform using Bayesian classification methods for classifier training and classification process, according to the trash mail filtering system evaluation verify whether the improved algorithm to improve the classification performance.

Related Dissertations

  1. Word Sense Disambiguation Corpus Automatic Acquisition,TP391.1
  2. Research on Text Classification Based on Biomimetic Pattern Recongnition,TP391.1
  3. Feature Extraction, Selection and Combination in Lipreading,TP391.41
  4. Multi-Sensor Information Fusion and Its Applications on Wearable Computer,TP202
  5. Yellow phosphorus storage tank area of Safety Evaluation,TQ126.317
  6. Research on Peer-to-Peer Traffic Identification Algorithm Based on Cluster Analysis,TP393.02
  7. Mapping Metabolic QTL and Constructing Metabolic Network in Perennial Ryegrass (Lolium Perenne L.),S543.6
  8. Research on Feature Selection and Construction in Emotion Speech Recognition,TP18
  9. Compressive Sensing of the Speech Signal and Its Application in Speech Coding,TN912.3
  10. Researches on Urban Rail Transportation Operation Management System Test and Evaluation Method,TP311.52
  11. Based on Data Distribution Characteristics of Text Classification,TP391.1
  12. Civil Protection of Internet Privacy,D913
  13. Research on Topic Clustering Model of Socail Tagging Based on Bayesian Theory,C93
  14. Risk Assessment of FPSO in Off-loading Operation,U698
  15. Research and Implementation of a Dynamic Feature Selection Method for Vehicle Recognition System,TP391.41
  16. Algorithm and Implementation Based on Non-rigid Medical Image Registration,TP391.41
  17. Research on Face Recognition Based on AdaBoost Algorithm,TP391.41
  18. Research on Feature Extraction, Selection and Classification Algorithms for Pulmonary CAD,TP391.41
  19. A Nonparametric Bayesian Method to Estimate Ordered Probit Model,O212.8
  20. Sensing Data Flasification Attack Defense for Cooperative Spectrum Sensing Schemes in Cognitive Radio Systems,TN925
  21. Image Spam Detecting Based on Combinatorial and Statistical Classifier,TP391.41

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Computer network > General issues > The application of computer network > E-mail ( E -mail )
© 2012 www.DissertationTopic.Net  Mobile