Talks
Speakers
Events
Topics
Sign in
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
Kiba ETL (http://www.kiba-etl.org) is a lightweight, generic data processing framework for Ruby, initially released in 2015 & now in v2. In this talk, I'll highlight why Kiba was created, how it is used for low-maintenance data preparation and processing in the enterprise (illustrated by many different use cases), why and how the version 2 (leveraging Ruby's Enumerator) brings a massive improvement in authoring reusable & composable data processing components, and why I'm optimistic about the future of data processing with Ruby. RubyKaigi 2018 https://rubykaigi.org/2018/presentations/thibaut_barrere
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
The video titled "Kiba 2 - Past, present & future of data processing with Ruby" features speaker Thibaut Barrère at RubyKaigi 2018, where he discusses the Kiba ETL framework for Ruby. Kiba, which means 'fang' in Japanese, serves as a lightweight, generic framework for data processing, facilitating easy data management and transformation within the Ruby ecosystem. Barrère explains the framework's development, its advantages in enterprise settings, and the substantial improvements introduced in Kiba version 2, which utilizes Ruby's Enumerator for a more efficient authoring process. The presentation encompasses the following key points: - **Understanding ETL**: Barrère defines ETL (Extract, Transform, Load) as a fundamental data processing concept, explaining its visual representation as a pipeline where data flows through different stages. - **Kiba's Origins**: He recounts his journey into data processing with Ruby, starting with the Khalid Act Veracity gem, which inspired the creation of Kiba in 2015 to address issues of maintainability and complexity in data pipelines. - **Core Functionality**: Kiba operates using three critical components—source, transform, and destination—allowing users to create customizable data processing scripts with simple, reusable patterns. - **Case Studies and Usage**: Barrère illustrates practical applications of Kiba, such as micro-batching for near-real-time synchronization, multi-step batch processing for enterprise aggregation, and the importance of automation in internal tasks. - **Version Two Improvements**: The enhancements in Kiba v2 focus on sustainability and ease of use, reducing developer burnout and ensuring the ongoing viability of the framework. - **Conclusion and Outlook**: He expresses confidence in Kiba's future, noting its unique value in Ruby for data processing compared to other languages like Python, while inviting questions and discussions about specific use cases and potential improvements. **Key Takeaways**: - Kiba offers a manageable solution for data processing with an emphasis on clean code and reusable components. - It supports flexible data handling practices, catering to various enterprise data processing needs. - The framework continues to evolve, aiming for user accessibility and operational sustainability, keeping pace with developer demands.
Suggest modifications
Cancel
