Dissertation > Excellent graduate degree dissertation topics show

Research and design of distributed processing framework based on cloud computing

Author: YangYiBin
Tutor: JiangChunHua
School: University of Electronic Science and Technology
Course: Software Engineering
Keywords: Cloud Computing MapReduce Job Scheduling
CLC: TP311.52
Type: Master's thesis
Year: 2011
Downloads: 550
Quote: 1
Read: Download Dissertation

Abstract


With the rapid development of e-commerce, the data, relating to international trade, also turn out to be a boom. In order to resolve some important technological problems in international trade, this thesis designed the distributed processing framework based on cloud computing to meet the requirements of storage management of mass data, disaster recovery, quality assurance of information services, efficient business information statistics, electronic documents conversion. This design not only takes good advantage of large-scale computer cluster of storage and computing resources, but also makes it convenient to expanse the clusters with low cost. It establishes a good technical support for international trade cooperation between enterprises and information exchange.This distributed processing framework is designed with hierarchical architecture, namely, hardware resources management layer, business logic layer, user interaction layer, bottom up. The three layers are independent of each other, the lower layer provides function calls interface to the upper, at the same time, the change of layer will not impact on other layers, so the framework can easily be improved and expanded. Hardware resource management layer uses Hadoop which is an open source cloud computing framework to manage the cluster hardware resources; the Spring open source framework technology and MapReduce programming techniques are applied to business logic layer, which mainly makes use of the underlying user interface to realize user management, document management, business information statistics, e-document conversion, programming interface calls and other business functions; user interaction layer, using Struts2 architecture, coupled with a variety of Web advanced technology, is responsible for the cluster interaction with the user, according to the user requests to call the appropriate business logic layer function modules.Considering the requirement of framework for multi-user and the demand for more short jobs, this thesis have done some deeply researches with MapReduce programming model of job scheduling approach. Through a large number of experiments and study of existing job scheduling algorithm, combining with features of the project itself, this thesis designed two time-driven queue job scheduling algorithm. The algorithms divides the job queues into long job queue and short job queue, which meets the needs of the short job queue, as well as avoids a long job occupying the drawbacks of cluster resources for a long time, and to some extent, realize the parallel operation and improve the utilization of cluster resource. Set last executive deadline, the job that meets the execute time can get the highest priority to handle firstly, and can get more resources, well meeting the user’s real-time needs. By delaying the task that can not be temporarily assigned to the data nodes resource for a period of time, we will achieve a better localization of data to improve the efficiency of the cluster.Finally, we had made a lot of experimental validation and comparison for the optimizations and improvements mentioned in this thesis. First confirmed that rational allocation and effective parameter can enhance the performance of the framework, and then confirmed the algorithm designed in this thesis is more superior to Hadoop default algorithm.

Related Dissertations

  1. The Research of Dynamic Trust Model on Cloud Computing Platform,TP309
  2. Several studies for scheduling problem,O157.5
  3. Cloud-based digital library service model,G250.76
  4. Design and Implementation of Online Shopping Prototype System Based on Hadoop,TP311.52
  5. Based on the study of resource management in the cloud environment of credibility,TP315
  6. Design and Implementation based the Google platform promotional modules commodity module,TP311.52
  7. Research and Implementation of job scheduling algorithm in the Linux cluster environment,TP301.6
  8. The Operation and Maintenance of ITIL Based on Cloud Computing,TP311.52
  9. The Research of Software Service Platform Based on Cloud Computing,TP311.52
  10. Virus Detection Technology Based on Artificial Immune,TP393.08
  11. Web application system design based on Google's cloud computing platform and,TP393.09
  12. An Intrusion Detection System for High-Speed Networks,TP393.08
  13. Hadoop data center deployment and tracking systems research,TP308
  14. Linux-based platform LAN Cloud Monitoring System Analysis and Implementation,TP311.52
  15. Logically oriented virtual domain multi -level access control system,TP309
  16. Fault Tolerance for MapReduce in the Cloud Environment,TP302.8
  17. Aircraft assembly moving assembly line job scheduling optimization,V262.43
  18. Simulation Resource cloud storage technology research and implementation,TP333
  19. Cloud data storage technology research homeostasis,TP333
  20. Cloud-based computing and immune reconfigurable embedded systems research network,TP368.1
  21. Cloud-based distributed real-time database performance data storage and retrieval mechanism,TP311.13

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer software > Program design,software engineering > Software Engineering > Software Development
© 2012 www.DissertationTopic.Net  Mobile