00:00:00.380
Ladies and gentlemen, I'm Yuki No Riki, and I would like to introduce our last speaker of the day.
00:00:05.400
This is John Mettraux. He is a chief member of Hiroshima.rb, which is a Ruby community in Hiroshima. We are very happy to have RubyKaigi here in Hiroshima, and I'm also from Hiroshima. I'm a local organizer here, so welcome to Hiroshima!
00:01:18.090
Hello, this slide contains a quote I've taken from a blog, but unfortunately, I cannot find the exact quote anymore. I know the blog and the person, but I can't recall the quote. So beware, programmers will invent languages, and I think that's normal. Our job involves extending languages at the very least, and we take pride in inventing languages.
00:01:43.619
As a programmer, I am guilty of inventing languages, and today I'll show you one language I've invented along with the interpreter that goes with it. My name is John. I'm from Switzerland, and my surname is Mettraux, so it sounds French. I come from western Switzerland, where they speak French. I'm a programmer, and you can find me under the username JMettraux almost everywhere. Perhaps you know my work from something that's quite old now, from 2007-2008—Rufus Scheduler.
00:02:10.950
It's a thread-based Ruby block scheduler where you specify when you want something to happen, encapsulating that request in a block. It works, and I have been maintaining it for quite a while, and I still enjoy it.
00:02:26.910
While I may have been called a chief member, I think the title is humorous. Let’s give back to Caesar what is Caesar's: I am simply a temporary organizer trying to steer Hiroshima.rb back on track. We try to meet at least once a month at West Lake these days, and the format is very simple—you show up, you speak for five minutes, and you can use slides or not.
00:02:56.009
If you don't talk, we give you Ruby quizzes, and you have to solve them. So far, everyone has participated with talks, and only once did we have to use the quizzes. So if you are in town, just join us via Canpass and share your insights, please. You're welcome!
00:03:40.470
Now, let's dive into Flor, the hubristic interpreter. As programmers, we tend to invent languages, and that's part of our hubris. I am a programmer with a considerable amount of overconfidence, and I want to solve problems. When we seek solutions to problems, we often turn to someone who has a problem, like a business professional or fellow programmer. Business people usually have funds, so they are more interesting to consult.
00:04:22.770
You may have seen in the talk description that, unfortunately, Flor just happens to be a workflow engine. The term 'workflow' has been used in various contexts—photographic workflows, git workflows, and so forth. However, nowadays, I hear the term used primarily in the context of personal workflows, meaning you as an individual have a set of steps to follow that you execute on your own.
00:05:13.920
This makes you your own workflow engine. When we need to address more business-like workflows, we start hearing about long-running workflows and business processes. Those are heavy words, but they signify the nature of what we're exploring today.
00:05:41.000
I began my obsession with workflow engines back in '99 while working as a Java developer. I developed a workflow engine for a health insurance company in Switzerland. Later, during the 2001 crash, I open-sourced my work to preserve it. In 2006, I moved to Japan and learned about Ruby, which I instantly loved. Prior to that, I had worked with Python and appreciated the scripting language feel.
00:06:01.710
In 2014, I had to stop working on Ruote after two years of satisfaction. Even Cookpad was using it within their Kuroko engine, which has since transitioned to Kuroko 2, moving away from Ruote. However, I felt proud because I could tell my wife that Cookpad was using my work, which she understood as it benefited her directly.
00:06:41.180
In 2014, I also started working with .NET for another company, but I wanted to keep my involvement in open source. Thus, I began porting Ruote and created a simpler version in C. I know, it sounds crazy, but I was envious of Redis. Salvatore Sanfilippo was developing this small tool called Redis, which everyone was using due to its simplicity: download, configure, make, and run.
00:07:03.380
At that time, I felt that some vocal members of the Ruby community were moving to Rust, and I was still deeply embedded in the Ruby ecosystem. I aimed to construct a workflow engine in C, which could enable small daemons to enact long-running business processes. In that process, I began gathering funds, planning to propose services surrounding that.
00:08:02.180
Then, a friend from the banking sector reached out to me. They wanted to build a mandate onboarding system and requested my help. However, the challenges of introducing a developer into their bank posed significant barriers, hence I returned to Ruby with JRuby. Thanks to the JRuby team for supporting that choice.
00:08:34.580
While developing the application for the bank, I initially worked with a straightforward lifecycle: requests were numbered, agreed, activated, and archived. Over time, complications arose where I wished for a workflow engine because the states transformed from a simple vector or table into a matrix of states. As these changes happen, I found myself resizing tasks not by a single entity but by what felt like a complex network.
00:09:39.910
Coinciding with my technical realizations, a business colleague informed me of our need for a sophisticated workflow, emphasizing their awareness of my expertise in workflows. To my surprise, they felt that my C implementation had to be entirely rewritten in Ruby. Imagine, an investment banker asked me to transition to code that I had developed in an entirely different language.
00:10:07.080
With his laptop, I started implementing the workflow engine, using my C prototype as a base. I had recently read a book stating that companies function like operating systems for business processes; it presented as a catchy concept, although I soon realized how many processes—like Pokémon—could not simply be digitized.
00:10:47.900
Despite my ambition, I struggled with how many processes exploded in complexity, particularly as they were fed to me like typical flowcharts. It became evident that visualizing the implementation of these workflows required different techniques than the traditional state-based approach.
00:11:07.230
In 2012, while working in a startup in the U.S., I was also engaged with orchestration tasks. As we discussed ways to visualize workflows, I witnessed a business personnel developing BPMN diagrams—augmented with process modeling—without any programming intervention. It struck me how significant the gap between programming and operational expectations had become.
00:11:51.060
The gap emphasized a disconnect within the startup space; developers were immersed in technology while business personnel had immediate needs to resolve. Meanwhile, they wouldn't necessarily concern themselves with sharing technical issues, preferring instead to purchase external software to manage them.
00:12:27.500
So, if you appreciate BPMN—and I know there are people in your company who do—you may find these BPMN-based gems currently being developed by the Trailblazer group significantly beneficial.
00:12:51.390
Business Process Modeling (BPM) is perceived as a discipline. It is crucial because these diagrams serve as the sometimes-overlooked key to adequate workflow execution—the implementation of business processes via correct control structures.
00:13:17.560
For implementation consistency, those diagrams tend to find their way into Rails controllers and Sinatra applications, making them essential to our roles as programmers.
00:13:44.270
However, these conversations—these models—often aim to create a seamless operating system for business processes. I began developing these process definitions, written in code rather than graphical formats, akin to a Lisp representation. Yet, akin to orchestrating musical flows, I wanted to manage the versions across tasks while ensuring those process definitions could be executed simultaneously.
00:14:33.400
One example of these processes is captured within the output of a shell tool—the Flor command—to display currently executing processes. As I ran these commands, I witnessed the various tasks waiting for responses from other participants, thereby allowing me to demonstrate how they interrelate.
00:15:03.220
As we observe these tasks, we can manipulate them: we can launch new processes or cancel existing ones. My goal is to provide a user-friendly experience through this system, ensuring even tasks can be interrupted or modified while conforming to underlying business rules.
00:15:38.860
For demonstration purposes, each instance of the system is dynamic: for example, if two processes are active and I modify the workflow to optimize interaction between them, I can significantly increase their efficiency.
00:16:21.830
Occasionally, some tasks could require urgent adjustments, all while still operating within collaborative workflow parameters. I aim to show you that while design control remains powerful, it can also allow real-time adjustments, keeping process flow optimized and efficient.
00:17:02.389
So let's dive deeper into the command line tool for Flor. As we go through, let's visualize the abstract syntax tree, meticulously created for each operation but not confined to traditional parameters. Through this, the operational pace can be adjusted, allowing for optimal functionality, particularly in scenarios requiring quick responses.
00:18:04.330
Looking back, humans are often the slowest components in these workflows. We continuously tweak processes, reactively evaluate whether a task deserves more immediate attention.
00:18:43.390
Take, for example, the implementation of a cron task. With proficiency, we can instill timing parameters that dictate task initiation or the management of multiple executions occurring simultaneously. Our tasks might evolve, with workflows developing to reflect new procedural changes—a constant discursive dialogue between programming and business needs.
00:19:28.230
Our objective is always to make these flows accessible, promoting efficient organization and ensuring no hindrance to operational speed is required, even when executing numerous tasks.
00:20:07.350
To reiterate, managing multiple parallel processes while still upholding the operational integrity of ongoing workflows is vital. Thus, let's examine the presented example, which illustrates modifying a flow for concurrent task achievement, ensuring that they effectively balance their outputs and maintain alignment.
00:20:50.230
As I demonstrated handling multiple executions, it became apparent how systemic management entails not only development but persistent refinements to guarantee continuous workflow enhancement.
00:21:30.860
In conclusion, this system widens our perspective on how workflows can cater to distinct functional expectations while emphasizing a core programming application embedded within business-oriented demands.
00:22:05.730
From tracking performance to systematic debugging, I believe this workflow engine can accommodate diversified tasks just as easily, offering transparency while affirming various capacities for role assignments and task allocations.
00:22:39.970
The beauty of this system lies in its versatility, allowing us to tailor executions conforming to evolving operational demands while maintaining clarity for each function embedded within the overall product offering.
00:23:09.590
In essence, with this framework, the intention is to create tools that address not only technical inefficiencies but also the need for clarity in organizational objectives and work processes.
00:23:44.760
I find immense gratitude for creating this community intent on fostering easy interaction among various users and inviting questions that help clarify unfamiliar organizational workflows.
00:24:21.640
I welcome any inquiries from the audience as we delve into planning out future expansions and collaborations that could broaden our common understanding.
00:25:08.500
Thank you for your attention! Now, are there any questions or comments from you?
00:25:37.320
Thank you for your talk! As someone new to workflow engines, I am only familiar with state machines used in my company. I wonder about the differences between them: when might it be appropriate to use a workflow engine instead of state machines?
00:26:08.800
I appreciate your question! For those accustomed to state machines, I recommend you continue to use them as they may suffice for your needs. Personally, I prefer to combine state machines that are explicitly associated with entities, layered beneath a workflow engine that manages the interactions among diverse entities.
00:26:51.590
When dealing with complex business processes, you will find adaptable efficiencies by transitioning to this broader concept of using a workflow engine.
00:27:22.520
Thank you! Do you have any more questions or comments?
00:28:15.100
Yes, can I use Flor to handle many tasks as integrated sub-task queues within another web application?
00:29:29.590
Yes, you can write the Tasker that puts tasks into background jobs systems like Rescue or Sidekiq, which are excellent for managing many jobs efficiently and executing them at specified intervals.
00:30:12.890
Feel free to connect with me at our next Hiroshima.rb meeting if you'd like to continue this discussion!
00:30:48.180
I appreciate your attention today! Thank you very much!