|
The web of public opinion on the Internet around the social event of the intermediary occurrence, development and change, the people on the social and political attitudes of social managers and held it to the forum, blog, news thread as the carrier. Due to the freedom of public opinion and the randomness of the network, more and more Internet users are happy with the forum, blog, news thread to express their views and communicate with each other. With Internet users height advantage of an open and free to speak their minds on problems in social development, in a very short period of time to arrive at a consensus, induced actions affect society. However, due to the limitations of the Internet users experience and level of thinking and understanding, it is difficult in a short period of time to sort out things behind the complex social and psychological motivation, irrational and overly subjective sound is often likely to prevail, from a series of incidents, Internet users through the network of public opinion broke out energy can not be ignored. In today's Internet growing influence, the Internet has become the ideological and cultural centers and public opinion amplifiers, the web public opinion has become more and more important. The party and government organs, enterprises and academic institutions are more and more attention to the monitoring of the Internet public opinion, research and guidance. We must fully understand the social influence of the Internet on behalf of the emerging media. Therefore, fast and accurate for effective monitoring is imperative the web public opinion, and the web of public opinion. Information systems web data with a massive, diverse, heterogeneous, dynamic change characteristics. Rely on manual methods which makes it difficult to deal with massive web information collection and the web public opinion found. In order to solve the above problems, web information collection and web of public opinion found that the learning and research. The main contents of the study include: web crawler technology, parallel computing technology, data partitioning technology, web public opinion found the technology. First of all, on the basis of existing reptiles technology research, design web crawler system, a change the policy, the reptiles mainly Universal reptiles crawling and can change the crawl time and crawl, so that the reptiles more friendly and efficient use of network resources. Then, found on the web public opinion research, proposed a classification based on the two-tier structure, the test proved that the classification algorithm proposed in this paper has better accuracy and practicality. At the same time, the use of traditional agglomerative hierarchical clustering algorithm parallelization achieve data classification method based on maximal frequent word sets. Then, data classification method based on maximal frequent word sets HADOOP platform implementation, testing results HADOOP has obvious advantages in dealing with large-scale data. Finally, according to the above-mentioned study, designed and implemented a web of public opinion found that the system, the system is set to change the policy of the reptiles and web public sentiment is found in one, and has a friendly interface and effective collection of information on the web, and more accurate to find that the web of public opinion.
|