Dissertation > Excellent graduate degree dissertation topics show

Research on JVM-based Integrating Framework for Large-Scale Statistical Computing in R

Author: CaoJie
Tutor: XieXia
School: Huazhong University of Science and Technology
Course: Computer Software and Theory
Keywords: Massive data Statistical computing Integrating framework R language JVM-based
CLC: TP311.52
Type: Master's thesis
Year: 2012
Downloads: 82
Quote: 0
Read: Download Dissertation

Abstract


Data explosion is experiencing in many domains such as astronomy, informationretrieval, and social network. The knowledge buries in these enormous datasets is soinvaluable that the ability to apply sophisticated statistical analysis methods to this data isbecoming more and more essential. As the most popular statistical language, R providesrich functionality for data analysis, but simply fails when the data becomes too large. Weproposed a generic framework JRBridge which can integrate R and JVM-based opensource computational infrastructures. Integrating R with Hadoop on this frameworkimproves the ability for large-scale statistic computing in R.In order to benefit from the lastest research achievements on computing frameworksand programming models for large scale data process, integrating R language withcomoutational infrastructures is our research thinking. With detailed analysis on how tointegrate R with popular open source infrastructures, we proposed SF and UDF integrationmodels. As most of open source computational infrastructures are JVM-based and thedefault APIs are written in Java, in order to make it easy for integrating, we designedJVM-based framework JRBridge which can integrate R and JVM-based open sourcecomputational infrastructures. JVM-based R language interpreter can interpretive executethe embedded R code in Java; Java Class Loader and Executor in R with jload,import and$operator make it possible to call method in Java library; R2J and J2R type convertersautomatically perform type conversion in the presence of context switching. With abovecooperating mechanisms and the plugin for integrating R and Hadoop, it is easy to handlelarge-scale data statistical computing in R.With the HDFS plugin, it brings a way to store and access datasets with millions ofobjects in HDFS. And with MapReduce plugin, it brings a natural environment to codeMapReduce algorithms in R. In the Hadoop clusters with5worknodes, the consumed timeof wordcount in JRBridge is nearly reduced to1/7of original R. The experiment resultshows that JRBridge scales linearly with the size of the dataset and thus provide a scalablesolution for large-scale statistical computing in R.

Related Dissertations

  1. Research and Implementation on Knowledge Representation of Gene Regulatory Network,Q78
  2. A Study of the Application of TBLT to English Teaching in Rural Middle School,H319
  3. Research on Compression, Operation and Query Processing Methods of Massive Datasets,TP311.13
  4. Research on Structure Transition Technology for SMT,TP391.2
  5. Word Sense Disambiguation Corpus Automatic Acquisition,TP391.1
  6. Implementation of Data Compression, Operation and Query Processing System Based on BAP,TP311.13
  7. Design and Simulation of Flight Management Computer System CDU Unit in Flight Simulation,TP391.9
  8. The Task-Based Teaching Approach and Its Application in Senior English Reading Class,G633.41
  9. A Study of Cross-cultural Pragmatic Failure in Senior High School English Teaching,G633.41
  10. Research on Language and Literature Group Consortium Teaching in Junior High School,G633.3
  11. Army logistics material management system design and implementation,TP311.52
  12. The Research of Consistency between the Tests and the Curricular Standards,G633.3
  13. On the Secondary School Language Teaching in the Lead Design,G633.3
  14. Reform and opening up three decades PEP high school Chinese teaching materials Review and Reflection,G633.3
  15. A comparative study ( early ) foreign language textbook \,H195
  16. On foreign language lessons in implementing a comprehensive task-based pedagogy,H195
  17. Korea Research Report of Chinese primary and secondary school classroom,H195
  18. A Study on Application of Task-based Language Teaching in English Pronunciation Teaching of Junior Middle School,G633.41
  19. The Research Oftibetan Supporting Technology on Tibetan and Chinese Distance Education Platform,TP391.6
  20. XML-Based Heterogeneous Data Integration Technology Research and Middleware Development,TP311.52
  21. China and Russia Chinese teaching methods compare,H195

CLC: > Industrial Technology > Automation technology,computer technology > Computing technology,computer technology > Computer software > Program design,software engineering > Software Engineering > Software Development
© 2012 www.DissertationTopic.Net  Mobile