Dissertation > Excellent graduate degree dissertation topics show

Rcscarch on Construction of ETL Cluster Model Based on Task Scheduling

Author: WangChunYang
Tutor: ZhaoShuLiang
School: Hebei Normal
Course: Applied Computer Technology
Keywords: distributed ETL tool cluster system task scheduling load balancingalgorithm of heterogeneous ETL cluster discrete particle swarm optimization
CLC: TP311.13
Type: Master's thesis
Year: 2012
Downloads: 28
Quote: 0
Read: Download Dissertation

Abstract


With the expending of the enterprise size and their business, the dataenvironment which they are facing with becomes more complex. ETL, extraction,transform and load, is an important part of data warehouse building and has taken alarge account of workload. The study on how to improve the ETL processing capacityhas gradually become the research focus of academics and received someachievements. These model theories of ETL only involve some simple factors of dataenvironment, so they cannot do well with multiprocessors under the distributedenvironment.This paper starts from the design of ETL tools model and the task schedulingstrategies, some ETL data operation and efficiency problems are solved which are indistributed data environment. The main work and innovations are described in detailsas follows:1. From the perspective of the entire implementation process of the distributedETL, this thesis proposes an improved distributed ETL model——ETL cluster modelbased on task scheduling and makes the model realized. This model consists ofworkflow generation module and task scheduling module. The introduction of taskscheduling module makes up the limitation which is the previous theory only paysattention to the workflow generation and ignores the workflow execution. The clustermanagement of processors improves the autonomy of system and reduces thedifferences between the data sources and network.2. The ETL cluster model based on task scheduling joins the managementfunction of ETL processors. The implementation of this function enhances thestability and reliability of system which is neglected in previous theory of ETL model.3. Around the problem of processor heterogeneity, we have to resolve the issuefrom two aspects, namely, heterogeneity of hardware configuration and software. Forthe hardware heterogeneity, we balance the task execution of processors in cluster by using load balancing algorithm of heterogeneous ETL cluster which reduces theimpact of cluster heterogeneity on the implementation of the ETL work and makesfull use of processor resources. Combination of Web services technology is used tomake the ETL implementation platform-independence which would resolve theheterogeneity of the processor software platform.4. The heuristic algorithm is used to study the task scheduling optimizationproblem of the distributed ETL in this paper. Considering the characteristics ofdistributed ETL tasks, we apply discrete particle algorithm to the distributed ETLtools. Experiment results show that the application of this algorithm has theadvantages and feasibility.

Related Dissertations

  1. Research of Scheduling Algorithm Based on Hybrid Adaptive Genetic Algorithm in Computing Grid,TP393.09
  2. The Sdudy on Grid Task Scheduling Algorithm and Its Modeling and Simulation with Colored Petri Net,TP301.1
  3. Kernel Analysis and Application Research of Embedded Real-time Operating System MQX,TP316.2
  4. Research on Task Scheduling Strategy of Cloud Computing Based on MPSO Algorithm,TP3
  5. Research and Implementation of job scheduling algorithm in the Linux cluster environment,TP301.6
  6. The Research on A Scheduling Algorithm for Real-time Tasks in Multi-core Systems,TP332
  7. Desgin and Study on Multi-node Hot-standby High Availability Cluster Software,TP311.5
  8. Design and Realization of Distributed Algorithms Based on Intelligence,TP338.8
  9. Design and Implementation of Market Benchmark Reference System,TP311.52
  10. Flight Control and Task Scheduling of Small-scale Unmanned Helicopter,V249.1
  11. Enterprise Manufacturing Execution System technology of a product applied research,F273
  12. Cloud-based Software Testing Services Research,TP311.53
  13. GALS -based multi-core interconnection and task scheduling strategy,TN47
  14. Multi-core parallel cluster environment GeoComputation execution time prediction technology research,TP338.6
  15. Cloud computing task -dependent mechanism of dynamic parallel scheduling,TP3
  16. Research and Implementation of the Runtime Configuration & Management Technology for Parallel Simulation,TP391.9
  17. Massive Video Conversion Platform Design and Implementation Based on Cloud Computing,TP311.52
  18. Research and Application on Design Resource Dynamic Scheduling Faced to Design Reuse,TB47
  19. Grid Task Scheduling Algorithm under the Improved PSO Algorithm,TP393.09
  20. Research and simulation of grid job scheduling algorithm based on the expectation trust benefit function,TP393.01
  21. Research on Reconfigurable Task Scheduling Algorithms Based on LRSS,TP393.05

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer software > Program design,software engineering > Programming > Database theory and systems
© 2012 www.DissertationTopic.Net  Mobile