Dissertation > Excellent graduate degree dissertation topics show

Research on Designing and Application of Topical Spider

Author: DuYiPing
Tutor: GuNaiJie
School: University of Science and Technology of China
Course: Computer System Architecture
Keywords: Search engine Web crawler Topic Search Link context Feature Extraction Weight calculation
CLC: TP391.3
Type: Master's thesis
Year: 2009
Downloads: 229
Quote: 5
Read: Download Dissertation

Abstract


With the constant expansion of the Internet, network users increasingly rely on search engines and other web tools to get the information they want from the network. However, general search engines face growing network size as well as people on the need for personalized information retrieval is often inadequate. Topic search engine, also known as vertical search engines through search application limit on a particular theme, providing personalized, professional search services, improve service quality, but also saves computing and network resources. Papers around the web and theme feature extraction method, feature weights to calculate the impact of the performance of the method as well as the length of the link context of the topic Web crawler to carry out the following research: classic MI mutual information method tend to choose a low entry CHI statistical methods tend to choose high-frequency terms, EXM propose a new method, which added to the within-class word frequency and degree of dispersion factors, the combination of the two. The experimental results show that the new method can effectively enhance text classification accuracy rate of 5-12% can be used for web crawler theme feature extraction. Rate advantage in a binary feature extraction algorithm OR due to neglect of the word frequency factors in the poor performance in the number of low-dimensional case, proposed EOR algorithm. The algorithm introduces the class word frequency and degree of dispersion factors enhance the low dimension text classification accuracy of about 5%, can effectively reduce the dimension of the text vector. The entry the EOR score and term frequency TF EOR feature extraction process calculated binding term weight calculation method TF-EOR. Experiments show that the method than the traditional TF-IDF can improve web crawler to crawl accuracy of about 4%, compared with the same type of TF-MI and TF-CHI, is also a slight advantage. Application of EOR feature extraction method and TF-EOR right to re-calculate transformation universal the web crawler Larbin, making it the topic Web crawler Tlarbin. Performance evaluation shows that the Tlarbin effective crawl topic pages, its extraction accuracy is superior to simple best-first search, Shark Search Larbin, prototype. On in Tlarbin platforms, the URL context as the URL degrees predictors of reptiles performance and quantitative analysis. The experimental results show that the URL context the reptiles performance in the case of considering the the parent pages correlation factors basically no effect, and does not consider the the parent pages related degrees, with the increase in the length to the continuous improvement of the reptiles crawl accuracy.

Related Dissertations

  1. Research on Automatic Detection Algorithm for Substructure Distress of Highway Pavement Based on SVM,U418.6
  2. ISAR Imaging Simulation of Space Targets and Target Recognition Based on ISAR Images,TN957.52
  3. Research on Feature Extraction and Classification of Pulse Waveform for Cholecystitis and Nephrotic Syndrome Diagnosis,TP391.41
  4. Application of Q-Learning in the Content-Based Image Retrieval Technology,TP391.41
  5. Research on Transductive Support Vector Machine and Its Application in Image Retrieval,TP391.41
  6. Research on Feature Extraction and Classification of Tongue Shape and Tooth-Marked Tongue in TCM Tongue Diagnosis,TP391.41
  7. Research on Visual Measurement for Spacecraft Rendezvous and Approach,TP391.41
  8. Research on the Image Real-Time Acquisition, Storage and Image Processing System,TP391.41
  9. Feature Extraction, Selection and Combination in Lipreading,TP391.41
  10. Multi-currency Notes Technology Research and Implementation,TP391.41
  11. The Research on Paper Currency Classification Method Based on Harr-Like Feature and Minimal Ball Including Samples,TP391.41
  12. Pavement Distress Recognition Based on Image,TP391.41
  13. Research on Visual Detection and Tracking of Mobile Robots,TP242.62
  14. Research on Fusion Algorithm of Hyper Spectral and High Spatial Resolution Remote Sensing Image,TP751
  15. An Approach for Identifying a Plant Resistance Gene Based on the Random Forest,Q943
  16. Tobacco Diseases Auto-Recognition Research Based on Image Processing Technology,S435.72
  17. Research on Nondestructive Detection Technology for External Qualities of Papayas Based-on Vision,S667.9
  18. Research on Identification System of Cashmere and Wool Fiber,TS101.921
  19. Research for Infrared Image Target Identification and Tracking Technology,TP391.41
  20. Research of Diagnosing Cucumber Diseases Based on Hyperspectral Imaging,S436.421
  21. Web search engine related technology research,G354

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer applications > Information processing (information processing) > Retrieval machine
© 2012 www.DissertationTopic.Net  Mobile