Talks

Trailblazer

Whenever Open-Source meets deep and profound debates about architecting software, and there's free beers involved, Nick Sutterer must be just around the corner. Say "Hi!" to him, he loves people.

Balkan Ruby 2018

00:00:09.590 Thank you! This was the most beautiful introduction ever. I was actually crying behind the poster!
00:00:14.969 As a professional speaker, I have to set my own timer because I'm a professional. So just an announcement before I start my actual talk.
00:00:20.400 I'm very, very humbled to invite you to the official secret after-party. It's going to happen at a place called Canal.
00:00:28.980 I didn't have Wi-Fi, so I couldn't translate it to funny characters. But I'm pretty sure you know the place, because it must be an awesome spot.
00:00:35.880 It's also a great idea to have the after-party after day one, and then have another day of the conference. It's always great!
00:00:42.360 I always love to talk at the last slot, because I'm a little bit hungover, but that's fine!
00:00:48.750 So my official conference talk title was 'From OOP to Functional Programming and Beyond'. You can see I'm really talented because I'm using Linux and don’t have Photoshop.
00:01:05.700 But I decided to make it even fancier and just call this talk 'Trailblazer' because apparently that's what I'm talking about.
00:01:12.990 Actually, I’m not only talking about Trailblazer, functional programming, and object-oriented programming; I’m talking about simplicity.
00:01:19.680 Yeah, because it's the last talk of the day and everyone is really tired. Are you ready for the after-party?
00:01:25.740 Whoo! That’s how I like it!
00:01:31.740 So, um, this is a bunch of my friends and me from two years ago. I can't find myself in the photo, but I’m there. This is about the age when I started to learn about programming.
00:01:50.810 I mean, I didn't actually know what I was doing, but I started to get interested in how to operate a computer instead of just playing with it.
00:01:58.140 I was one of the first kids in my town to have an Apple Computer Classic. You know, the little box that people now use for fish bowls.
00:02:05.700 I had only five games because at that time Apple wasn't cool. I had an Apple before it was cool.
00:02:15.430 So I started playing a bunch of computer games, all in black and white. For some reason, I wanted to program a computer game.
00:02:22.239 My dad bought me this book, and back in the day, a book like this cost about $200, and it was imported from America.
00:02:32.910 I didn’t even really speak proper English, but this was kind of my first programming book. I read 'Tricks of the Trade: Game Programming Gurus'.
00:02:39.300 It's pretty awesome, and I still have it somewhere.
00:02:46.329 Then I realized that programming actually means you have to learn how to program. The book was written in C++, which is perfect for a 9-year-old kid.
00:02:52.030 I started to learn C++ and concepts like variables, and if-then-else statements.
00:02:58.299 I remember exactly the day when I learned about functions. I understood that a function is a component I can reuse.
00:03:06.670 I got that, but I didn't understand why the arguments A and B were named that way. I thought you had to name them according to what the user of the function wanted.
00:03:14.530 Then I kept thinking about it. I was driving with my dad one beautiful November day, and he kept explaining what a function was.
00:03:20.530 At some point, it clicked for me. I realized that a function is a piece of logic not affected by anything happening outside of it.
00:03:29.080 You push something in, it does some stuff, returns something and nothing else changes. That really intrigued me.
00:03:36.000 I had no idea I would be standing on stages talking about components and object-oriented and functional programming.
00:03:43.930 But that was the moment when I realized cool programming is a lot about structuring.
00:03:50.360 And a function is kind of my best friend. Unfortunately, I made a big mistake and started learning Perl.
00:03:56.360 That was my first step into web programming. My friend asked me, 'Are you good with databases?' and I was like 'What?'
00:04:03.910 So I started to learn about web programming. Back then, you had to use PHP or Perl.
00:04:10.570 After about one or two years, I actually had a website up and running to sell boats and yachts.
00:04:17.890 It's not that I owned a lot of boats and yachts, but people would put their advertisements on it. It was like eBay for rich people.
00:04:24.240 I wrote it for rich people, but no one used it! That's kind of the story.
00:04:34.130 While I was doing Perl, I was still intrigued by the idea of components and having code not affected by other code.
00:04:40.250 I didn’t even know what ‘minimizing side effects’ meant at that time.
00:04:49.700 I had a really strict object-oriented way of designing my Perl code. Then I made another mistake and started to learn Ruby on Rails.
00:04:56.800 Ruby is an amazing language, especially when you come from Perl, because Perl is so ugly, and Ruby looks beautiful.
00:05:05.370 Ruby has a dot operator and not some weird arrow, and you actually have objects instead of having to bless some reference.
00:05:14.000 In Rails, my first problem was that I didn’t like how you only had three buckets for code: model-view-controller.
00:05:20.910 You throw validations, callbacks, and everything into models, and do the same in the controller.
00:05:28.960 It’s hard to see where the code is for updating things like a blog post.
00:05:39.890 If I ask someone, 'Where is the code for updating a blog post?' they say, 'Um, a little bit in the controller and a bit in my own service object.'
00:05:45.310 I was missing structure; I wanted a domain view, with a way to put all the code for updating a blog post in one place.
00:05:53.300 Rails is cool and easy to learn, but once it gets more complex, it’s actually not that simple anymore.
00:06:01.200 I started to work on a few gems; the sales gem was my first gem.
00:06:08.950 Back then, we didn't even have gems; I think it was just a Rails extension or some Rails plug-in.
00:06:19.130 What Sales does is help you create reusable widgets in Rails. You can encapsulate that code in the view for a reusable element.
00:06:27.460 It’s just like having a class and some code for that widget, and you can call that class with a helper method.
00:06:35.380 The cool thing was I could test that in isolation. I could test my partial or my widget in a unit test.
00:06:43.700 And that was pretty cool because I love testing. I love when I can test stuff in isolation.
00:06:50.920 This connects back to my fascination about functions; it made intuitive sense to me, not only to me but also to others.
00:06:58.200 Have you heard of DHH? Here’s an email from 2008 where he said he really liked Sales.
00:07:06.800 Of course, nothing evolved from that; DHH and I are still friends.
00:07:15.300 DHH is the guy who is famous for his emphasis on pushing everything into one class, calling it amazing.
00:07:23.200 He claims you can make millions by doing it—that's what they do in Basecamp.
00:07:30.000 Another gem I worked on is the Reform gem. It’s a gem that only sorts out validations with form objects.
00:07:39.170 If you have a form on your website, you define the fields and validations in that form's class.
00:07:46.130 Instead of pushing validations into the model and controller, you have all the validations in one class.
00:07:54.200 The cool thing about this is that you can test it; you can call that validation in isolation without the entire application complexity.
00:08:01.300 You just need to call the function, and it will give you a result object.
00:08:07.000 What I found was that while using those gems, I still felt lost at times in Rails codebases.
00:08:15.200 At the same time, Rails was telling people to push all code into one class, which can lead to complexity.
00:08:24.250 The more you separate components in your code, the simpler it gets. It isn’t simple to have one class with a million responsibilities.
00:08:34.250 This is when I started to think about an alternative framework, and I began working on Trailblazer.
00:08:41.500 The key insight of Trailblazer is that it does not replace Rails; we are not trying to make all the cool things of Rails null and void.
00:08:48.000 We’re extending the framework. If anyone has a good inspirational quote, please tell me.
00:08:55.200 In the first version of Trailblazer, we had very simple controllers that only did HTTP-specific stuff and rendering.
00:09:04.360 We called what we called an operation, and the operation took over the orchestration of the business code.
00:09:10.180 So controllers are very simple, and the models only handle what ActiveRecord does well: querying and writing to the database.
00:09:17.600 We can discuss if the writing part in ActiveRecord is good or not, but it's great for SQL abstraction.
00:09:27.490 But it’s not great for callbacks, validations, or domain code.
00:09:32.930 In Trailblazer, the goal is to have an ActiveRecord class that only does SQL abstraction.
00:09:40.490 Models are cleaned up; instead, we introduced a new structure called operations.
00:09:47.700 Every operation has only one responsibility: creating a comment, deleting a user, or archiving a blog post.
00:09:55.840 Many people were concerned that we were just creating another god object, and the operation would do all the code.
00:10:03.130 But the operation does not implement everything in one big chunk of code; instead, it orchestrates different stakeholders.
00:10:10.190 We connect with form objects, ActiveRecord, SQL, ROM, or whatever you want.
00:10:16.200 The operation acts as an orchestrator, not an implementation; it’s very beautiful.
00:10:24.350 Here's a version one example: the operation is a class. You see the name 'comment create'; it’s going to create a comment.
00:10:31.180 People were scared about creating new classes in Ruby; they thought it was going to be slow.
00:10:38.200 But this is object-oriented programming; if you need to solve something, you solve it in a separate class.
00:10:46.700 Ruby’s garbage collection is designed for small objects that die quickly, not for huge monolithic objects.
00:10:54.420 The funny thing was that the Trailblazer way actually sped up applications.
00:11:01.130 The idea of this operation is to have one public method, and you call that operation.
00:11:08.550 It runs whatever it does and returns you a result object with no other public methods.
00:11:15.230 It's like a function in functional programming; it allows you to avoid messing up system state.
00:11:22.640 The version one result object tells you whether it was successful or not.
00:11:30.020 You ask the operation instance if there was a mistake, and the cool thing is you can test it.
00:11:36.760 You can call that operation in a unit test, and it behaves the same in a controller.
00:11:43.400 It’s amazing, and it’s hard in Rails to replicate that without an integration test.
00:11:50.230 The idea was to take code into a separate class and make it testable, which I really liked.
00:11:56.120 Another amazing aspect of the operation was that you could use it as a factory.
00:12:02.890 Instead of using FactoryBot or whatever you use for fixtures, you would use the operation.
00:12:09.460 You could chain operations to set up application state, making everything simplier.
00:12:16.300 You’d have the exact application state you needed for testing.
00:12:22.630 By making things explicit, it actually got simpler. Then I had a great idea.
00:12:30.440 I started writing a book about the framework while working on it. It was horrible because as I changed the API, I also had to change the book.
00:12:36.890 The book was called 'Trailblazer: A New Architecture for Rails'—you don't have to buy it.
00:12:43.050 If you buy it, I will have more money to buy Rakia tonight, which is great!
00:12:50.110 People were mad because they thought the project must be finished because there's a book.
00:12:56.110 It didn’t solve all their problems; surprise, open source! They got really mad at me.
00:13:03.060 They thought, 'This book is going to solve all your problems automatically.' It wasn’t perfect.
00:13:10.350 But that was a crazy experience—writing while simultaneously developing a framework.
00:13:17.100 The main idea for Trailblazer was to have an operation reflecting a verb of your domain: creating, deleting, or archiving.
00:13:24.870 Initially, the operation had one method called process, where you’d do all the code.
00:13:30.270 That was a mistake because people wanted to customize what happens.
00:13:37.420 They would have to know the call stack, which was really stupid.
00:13:43.350 No one wants to learn the internals of a call stack and understand the method calling order.
00:13:49.540 Came from C++ and Perl, where everything is about objects and call stacks.
00:13:58.280 But it turned out to be the wrong design for an orchestration class.
00:14:04.750 The principle of having methods calling methods with super became complicated.
00:14:10.330 I found out this wasn’t a good architecture.
00:14:16.030 Ruby has great objects, but the way modules work may have issues.
00:14:22.370 So I started to consider moving into functional programming.
00:14:29.410 I learned that functional programming is simple: you pass input, you get output back with no side effects.
00:14:37.670 The cool thing is that nothing else changes beyond that.
00:14:45.200 Looking at the operation, we were already doing what a function does.
00:14:52.430 I didn’t recognize it as such until I learned more about functional programming.
00:15:00.880 It became apparent that operations are functions in their own right.
00:15:08.200 However, we still had this problematic call stack. I knew we had to simplify that.
00:15:15.000 In the next version of Trailblazer, we introduced a DSL to define steps.
00:15:23.680 This DSL allows you to declare what you want your business code to do.
00:15:31.150 Each operation builds a graph of steps in the background, allowing for conditional operations.
00:15:39.780 This was inspired by railway-oriented programming.
00:15:46.050 The idea is that this pattern allows the flow of control to be managed without difficult programming.
00:15:53.030 Via the DSL, operations can create a structured path for workflows that either work or skip certain steps.
00:16:01.630 I created an animation demonstrating this concept, and I hope to get some appreciation!
00:16:08.650 It simplifies the session’s workflow and allows programming through clear steps.
00:16:15.880 Using a DSL and visualized graphs allows us to represent operations in a much clearer way.
00:16:22.670 Users found it exciting to see steps in graphs instead of all the if-else structures.
00:16:29.140 The operation transforms the complex into something simple and visually clear.
00:16:36.420 Everyone liked using functions in a way that didn’t get convoluted with too many steps.
00:16:43.450 And still, they began to request more features after getting used to this.
00:16:50.750 It’s always after you make something easier that people want even more.
00:16:57.150 They suggested tracing features so they could visualize workflow paths.
00:17:03.520 Tracing is important because it shows the path an operation takes through different steps.
00:17:09.820 The challenge was I had to change a lot for this to work.
00:17:16.630 In version 2.0, we shifted from the process method to the DSL.
00:17:23.300 People didn’t like that we broke the API; they preferred to stick with the old way.
00:17:29.090 Working on open source often becomes a psychology experiment, because people resist change.
00:17:35.490 When you innovate, people push back, but later they often appreciate what you did.
00:17:42.410 In the new version, we included tracing so they could visualize inputs and outputs.
00:17:48.890 This feature helps to understand what is happening at every state, making operations better.
00:17:56.240 So we gave them longer-running workflows which could handle millions of requests.
00:18:02.830 Using the workflow engine, we gain control over what happens in our code.
00:18:09.530 We provided a visual editor to allow users to model their workflows better.
00:18:16.550 It allows you to link operations visually without having to rely on complex code.
00:18:22.400 Peter Wagner’s business process modeling notation (BPMN) aligns with what we did.
00:18:30.090 This is useful because the police department had numerous processes they needed to visualize.
00:18:37.150 They had tons of documents full of diagrams which didn’t correlate with the code.
00:18:43.570 By using BPMN, we can automatically generate required documentation from the code.
00:18:50.400 This process will cut down refresh cycles for these confusing documents.
00:18:56.640 Finally, models can create these diagrams, making everything run smoother.
00:19:03.800 Now we're seeing a shift towards a simpler, more visual programming style where we focus on smaller components.
00:19:10.990 You only work on the implementation while the control flow is handled by the framework.
00:19:18.950 As we wrap up this session, I encourage you to explore these new approaches within the Ruby community.
00:19:25.560 With all the new gems, Ruby can greatly benefit from the innovation surrounding functional programming.
00:19:32.090 Embrace simplicity, create components and abstractions instead of stuffing everything into one class.
00:19:39.740 Thank you for listening! Don’t forget to come to the after-party at Canal!