Talks
Speakers
Events
Topics
Sign in
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
http://rubykaigi.org/2016/presentations/masa16tanaka.html Pwrake: Distributed Workflow Engine based on Rake Pwrake aims at the high-performance parallel execution of data-intensive scientific workflows using multi-node clusters with 10,000 cores. In the design of Pwrake, I made use of existing powerful tools. First, Pwrake is implemented as an extension to Rake. In this talk, I show that Rake is so powerful that it enables portable definition of workflow DAGs comprised of many tasks. Second, Pwrake has an option to make use of Gfarm distributed file system for high-performance parallel file I/O. Also, I will talk about other studies on Pwrake such as locality-aware task scheduling. Masahiro TANAKA, @masa16tanaka Research Fellow at CCS, University of Tsukuba. The author of NArray, a numerical library for Ruby.
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
Suggest modifications
Cancel
