Talks
Speakers
Events
Topics
Sign in
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
Data streams (ex: logs) are becoming larger, while analytics in (semi-) real time is also becoming more important today. Moreover, there are many non-programmers who wants to write queries in this big data era. Norikra is an open source server software that provides "Stream Processing" with SQL, written in JRuby, runs on JVM. We can write and add queries for stream easily on Norikra, without any editors, compilers or deployments. I will talk about implementation of Norikra and its application in LINE corporation. Help us caption & translate this video! http://amara.org/v/Fqsv/
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
The video titled "Norikra: SQL Stream Processing in Ruby," presented by Tagomori Satoshi at RubyConf 2014, focuses on Norikra, an open-source server software designed for processing data streams using SQL within a Ruby environment. The presentation underscores the increasing significance of real-time analytics and the challenges posed by large, complex data streams. Norikra operates on JRuby and utilizes the Java Virtual Machine (JVM), enabling users to execute and manage queries seamlessly without reliance on heavy development tools or processes. Key points discussed include: - **Objective of Norikra:** Norikra facilitates query writing and management for non-programmers, making it accessible in the era of big data. - **Use at LINE Corporation:** Satoshi highlights how LINE Corporation handles a substantial volume of logs and metrics as it supports a messaging application with approximately 130 million users. This involves monitoring various sub-services and ensuring effective data management. - **Data Processing Needs:** The company’s analytics platform relies on collecting data from numerous servers and storing it in distributed systems like Hadoop's HDFS, which necessitates efficient and timely data processing to address operational issues. - **Comparison to Other Tools:** The speaker critiques existing solutions like friendly, which require cumbersome configuration and restarts for complex datasets. Norikra is presented as a more flexible alternative that supports dynamic query adjustments. - **Interactive Features:** Demonstration of how to install and utilize Norikra via command line, emphasizing its ability to handle JSON data inputs flexibly and process them using SQL queries. Integration with different output formats facilitates customizable data visualization. - **Real-time Application:** Norikra is employed in summarizing error logs effectively, enabling LINE Corporation to manage API interactions without overwhelming partners with too many notifications. - **Scheduled Analytics:** The platform also supports generating reports based on larger datasets, providing insights crucial for decision-making. In conclusion, Norikra is presented as a robust tool for stream processing, adaptable to real-time analytics demands and capable of integrating with other data platforms. The speaker encourages viewers to explore Norikra via its GitHub documentation, citing its potential to enhance data processing capabilities significantly.
Suggest modifications
Cancel
