Get an improved navigation experience with a Chrominium based browser.
Dismiss
Ruby Video
Talks
Speakers
Events
Sign in
Talks
Speakers
Events
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
http://rubykaigi.org/2016/presentations/masa16tanaka.html Pwrake: Distributed Workflow Engine based on Rake Pwrake aims at the high-performance parallel execution of data-intensive scientific workflows using multi-node clusters with 10,000 cores. In the design of Pwrake, I made use of existing powerful tools. First, Pwrake is implemented as an extension to Rake. In this talk, I show that Rake is so powerful that it enables portable definition of workflow DAGs comprised of many tasks. Second, Pwrake has an option to make use of Gfarm distributed file system for high-performance parallel file I/O. Also, I will talk about other studies on Pwrake such as locality-aware task scheduling. Masahiro TANAKA, @masa16tanaka Research Fellow at CCS, University of Tsukuba. The author of NArray, a numerical library for Ruby.
Date
Summary
Markdown supported
Suggest modifications
Cancel