Philip Arndt
Taking Refinery off the Rails
We're seeing more evidence that the future of Ruby may not be tied to Rails as newer micro-frameworks and purpose built libraries are released to mass appeal.
In this talk we'll explore what happens when we take the most popular CMS for Rails, "Refinery CMS", and rebuild it with Hanami and dry-rb, plus reliable Frontend frameworks and libraries rather than a mess of Rails, jQuery, and organic CSS.
We'll provide you with both gems and solid advice to allow you to refactor your own applications to take advantage of the solid and agile foundation provided by Hanami and dry-rb.
RubyConf AU 2017
00:00:09.920
Good morning everyone, welcome to our talk, 'Taking Refinery off the Rails.' I'm Phil.
00:00:17.580
I’m from a nuclear coalition in New Zealand, and I live in Christchurch. It seems there’s been some confusion as everyone thinks I live in Auckland or Wellington, but no, it's Christchurch. I’ve been involved in the Ruby scene for quite some time now and I'm quite interested in contributing to open source.
00:00:30.090
I gave a talk, I think it was two years ago at the last normal conference, about succeeding in open source. Thanks to the power of an anonymous CFC, I got back into it and I also really like to travel. With me today is my colleague Sam.
00:00:44.960
Sam and I worked together on building the Refinery prototype that we’ll be discussing this morning. Sam is primarily a front-end specialist and cares deeply about performance and board games.
00:00:58.079
The talk is structured as follows: first, we will explore Refinery's current position and how it is implemented, along with some of the technical debt we’ve encountered and the pain points we’ve run into. Next, we’ll look at the Hanami framework, which you may have heard of, along with dry-rb, to answer the question: is it a better alternative to what we're currently using?
00:01:13.200
Finally, Sam will talk about architecting a modern CMS that is API-first and integrated with Webpack. So, Refinery CMS is the topic today. It's a content management system, and while it may seem obvious that it manages content, it is built in a neat way. It is distributed as a few core Ruby gems and many extension gems, all written as Rails engines.
00:01:39.510
These engines hook into your Rails app and provide content management functionality, so you still have control over more advanced aspects of your site as you can write Ruby code and JavaScript. This allows users or customers to edit content without needing to redeploy the website, which is quite key for a CMS.
00:02:15.330
We call it a Rails CMS and I've just mentioned the most popular part. So, what do we mean by Rails CMS? Well, it only works with Rails. Given this, why did we choose to base it so heavily on Rails? Well, first of all, Ruby on Rails was around and it does a lot. We get many things for free and it allows us to be extremely productive.
00:02:39.470
It’s built for the '15-minute blog' and helps get you up and running quickly, supporting rapid feature development. This accelerated our popularity significantly because we had a lot of developers involved, which helped us expand the system. I've even had conversations with people here who use it, which is quite humbling.
00:03:11.760
Rails has a huge market share; for better or for worse, it seems like almost every Ruby gem is written for Rails. We hook them into ActiveRecord in some way to add magical special features you didn’t know existed. It’s very friendly for developers. You just put the gem into your gem file, run some commands, and voilà!
00:03:38.850
Rails is also actively maintained, which is a great advantage, but I found it to be quite a double-edged sword as a gem maintainer. New versions of Rails come out all the time. You might think major versions are the only issue, but the frequency of updates can make life quite challenging.
00:04:06.510
There’s an expectation that if your gem uses Rails, you will support the latest version immediately. If you don't have support within six hours of the release, your issue tracker fills up with requests asking where the Rails 6 support is. While it’s nice that people are interested, this expectation creates a significant burden on gem maintainers.
00:04:40.420
Thus, my angle for this talk is from a gem maintenance perspective. It’s not about writing web apps with Rails; it’s about maintaining software that many people use in the form of a library set that can be integrated into their own projects.
00:05:00.670
Front-end development has been quite tempting. We often rely heavily on Rails public APIs, and as it turns out, Rails has quite a few private APIs as well. You can access these effortlessly with some nice monkey patching.
00:05:23.920
This approach avoids doing actual work ourselves and leads to a tight coupling with Rails. This creates challenges from a gem maintenance perspective, as it complicates support across various releases.
00:05:49.120
So, where are we now? Historically, Refinery ended up being very coupled to Rails. While this allowed us to leverage many features, it also created a burden of support and maintenance.
00:06:08.440
This meant we accumulated a significant amount of technical debt. Each version of Rails that was released was quickly integrated, but we often rushed to turn failing tests into passing ones with the minimal effort allowed, given that we're working in an open-source context.
00:06:34.060
Sam and I began discussing ways to reduce this technical debt after I showed him the project and explained the challenges we faced. Rather than gradually refining the current solution, we decided to take a creative approach and try something new.
00:06:55.860
This led us to Hanami, which was previously known as Lotus. We thought it was a good opportunity to build on the foundational concepts of Refinery using Hanami. Initial impressions were quite positive; Hanami feels very Ruby-esque.
00:07:18.380
It feels natural and builds on top of Ruby with composability that reduces the need to dive deeply into learning the Hanami framework. The Refinery website states it follows the Rails philosophy, but this time we didn’t have to worry about that. Instead, we relied on our understanding of object orientation and Ruby.
00:07:47.560
Hanami encourages a proper design for database access and persistence. However, it does impose some restrictions—such as prohibiting direct database queries in views—that, surprisingly, are beneficial. This design forces you to think about your data handling and encourages writing repository methods.
00:08:16.900
This approach prevents the construction of hidden, undocumented, and untested bits of code, enhancing overall quality. This is somewhat similar to the intent behind Elixir's Exo library.
00:08:47.080
Moreover, Hanami promotes the separation of concerns within your application. Its various libraries, while operating independently, are remarkably complementary when used together.
00:09:01.670
The real benefit is that Hanami now utilizes dry-rb libraries. This collaboration showcases the Hanami creators' vision, as they were able to incorporate existing tools rather than building their own from scratch.
00:09:27.780
The advantage is that we can learn dry-rb technologies and utilize their abstractions directly within Hanami. These dry-rb components offer solid, first-class APIs, making for a more effective development process.
00:10:00.840
One of the coolest aspects of this setup is that we use the Dry Validation library for all of our validation needs within Hanami. This robust validation library is grounded in predicate logic.
00:10:28.050
Let’s consider a scenario where we need to post some JSON data to construct a page. The page is central to our content management system, and each content piece has a content type and value schema.
00:10:56.260
The value schema is essentially a nested hash with key-value pairs, including titles and slugs for easy retrieval. We use a params block that closely mirrors this JSON structure, defined using Ruby syntax.
00:11:11.900
This method enhances readability and makes it simple to understand the expected data structure for validation. Server-side, we can validate JSON data seamlessly, which is an invaluable feature for maintaining data integrity.
00:11:38.350
You might notice the absence of deep hashes and arrays, as the design is focused on predicates. The methods used at the end of lines return true or false based on the expected criteria, which simplifies the validation process.
00:12:04.190
We can also accommodate optional parameters, which is a rarity in many libraries. If a parameter is included, it must conform to the specified type, greatly facilitating error checking.
00:12:31.120
Furthermore, Hanami utilizes Ruby object meta programming, which enhances our data management. After some initial challenges with this new approach, I discovered its strengths and how useful it is for our projects.
00:12:58.160
This integration allows for seamless interaction with the Portable Sequel gem, simplifying the complexities involved with SQL queries. We now have access to features such as JSONB support, allowing for richer data management.
00:13:25.700
This enables us to write efficiently structured queries that can sift through JSONB fields in the database. What’s better than combining the strengths of a relational database with NoSQL capabilities?
00:13:55.050
This enhancement allows for rapid query formation, which is evident in our performance tests. For instance, we can now execute queries that rapidly search for items based on JSONB content efficiently.
00:14:20.990
Following up on performance, we utilize the entity mapping feature, allowing our implementation to align closely with the expectations set by Hanami's conventions.
00:14:56.970
From a gem maintenance perspective, it becomes apparent that combining Hanami with dry-rb technologies presents compelling advantages. I’ve found that maintaining library code is easier as it lives in a Lib directory, reducing unnecessary coupling.
00:15:29.860
This organization encourages thoughtful dependencies, as enumerating required files enables better understanding and modularity.
00:15:52.690
Moving forward, we would strongly consider relying more on dry-rb technologies instead of coupling tightly with another framework. The features of dry-rb can provide ample horsepower for our needs.
00:16:18.600
The bulk of web app code often falls within the Ruby application logic rather than needing to be tightly integrated with the web framework. We envision distributing our functionality through smaller, modular gems.
00:16:36.790
This modular approach gives users the flexibility to enable or disable specific features easily, promoting a more customizable experience.
00:17:02.380
The takeaway is that Hanami and dry-rb technologies provide an exciting path forward. I’m eager to hand things over to Sam for the remaining part of the presentation.
00:17:23.540
Thank you.
00:17:48.290
Oh, hi! This is my first conference talk, so it’s pretty exciting for me.
00:17:58.429
So awkwardly, I’m going to talk about front-end development, which includes JavaScript.
00:18:04.970
There are no jokes in my slides because JavaScript is serious business.
00:18:11.029
I'm going to discuss the front-end considerations we made while developing our prototype for Refinery, including design choices for a more modern CMS.
00:18:26.289
Our initial focus was to create an API-first CMS. During the architectural planning phase, we decided it should solely serve a JSON API. This was a significant improvement for Refinery, which previously relied on a mix of custom JavaScript and Rails forms.
00:18:57.070
The previous implementation created a lot of unnecessary coupling, making it challenging to add features or redesign aspects of the front end. The current dashboard of Refinery definitely needed a rework.
00:19:32.289
The rise of headless CMS solutions reminds me of services like Contentful, highlighting the benefit of hosting content types in one place for diverse consumer access. By designing Refinery as an API-first application, we can easily support multi-site frameworks.
00:19:57.130
Building a JSON API with Hanami was straightforward. I initially worried about relearning a new DSL, but it was surprisingly mature and user-friendly.
00:20:02.450
Creating the JSON API involved some simple configuration changes to our controllers to serve JSON instead of HTML templates, making the transition smooth.
00:20:29.940
Using dry validation to establish expectations for incoming data was wonderful for both front-end and back-end synchronization. The clear validation structure encourages seamless communication.
00:20:54.070
We decided against using Rails forms. Instead, we aimed to redefine our front-end strategy with React, driven by its reusable components. The decision was informed by the hype surrounding it but also by its capabilities.
00:21:08.700
We explored preact as well, which has the same API as React but is significantly lighter, promoting better performance—especially for users with slower connections.
00:21:25.600
Here’s a quick example of using JavaScript in our application. Though it might look complex to some, essentially, we can dynamically build forms based on a structure from our API.
00:21:51.590
Content types are defined by how users structure their stored content, such as blog posts. We can easily create various field types dynamically.
00:22:07.130
Now, regarding tooling in Hanami, we examined its asset pipeline—it mirrors the one in Rails, which can seem daunting.
00:22:27.550
The Hanami asset pipeline supports multiple formats like ES6 and CoffeeScript, but I found drawbacks, such as it lagging behind on new JavaScript features because it’s tied to an outdated Ruby Babel transpiler.
00:22:56.450
Bridging JavaScript tools with Ruby is often frustrating. I would recommend avoiding bridging gems where possible; they can impose additional burdens on maintainers.
00:23:23.150
Instead of utilizing Hanami’s asset pipeline, we integrated Webpack, sidestepping the cumbersome points of asset management. We don’t need an asset pipeline when Webpack can handle that more efficiently.
00:23:53.870
Running Webpack alongside Hanami proved to be straightforward. We can build production-ready assets and smooth development transitions between them.
00:24:19.680
Here’s a quick look at our Ruby helper for integrating built assets. It loads a manifest file generated by Webpack into memory, allowing us to easily manage script tags for assets.
00:24:43.210
This simple modular setup has minimal maintenance overhead. While we manage substantial codebases daily, a brief helper script remains manageable.
00:25:04.070
The benefits to this approach are clear—we can upgrade Webpack independently of the rest of our application without overburdening our development processes.
00:25:32.000
In conclusion, we’ve crafted a proof of concept where we provided a persuasive case for merging Hanami and dry-rb technologies.
00:25:49.840
If we can do it over again, we’d embrace dry-rb principles much sooner, establishing a clear boundary between our library and web solutions.
00:26:15.550
For web developers, we’re genuinely excited about what Hanami offers. It streamlines the development process and forms a solid foundation built on dry-rb technologies.
00:26:36.640
For anyone familiar only with Rails, trying out these tools offers an opportunity to rediscover the excitement of programming that can diminish when focusing solely on a single framework.
00:27:09.350
This exploration of new technologies brings back the passion that many developers experience when they first start coding.
00:27:39.110
We envision an exciting future for Ruby and JavaScript, where each language plays to its strengths, facilitating a richer end-user experience.
00:28:04.990
Thank you very much for your time.
00:28:38.490
So, we’re running a little early, which is fine since I may have messed up the schedule. If it’s okay, we can take a couple of questions.
00:28:55.510
Feel free to ask anything!
00:29:05.490
Phil, you mentioned using separate endpoints. Would you envision hosting different applications on multiple servers for load balancing?
00:29:16.970
Absolutely! Many setups use environment variables to boot particular apps on specific servers. While a monolithic approach is common, it’s possible to run different endpoints across various applications.
00:29:37.240
Each application leverages underlying library code, which are just Ruby gems in your gem file. This approach encourages reusing the same code across different services.
00:29:59.020
Given your perspective on dry-rb and Hanami, do you think it would detract from your design concept to couple with dry-web, or does that fit under the umbrella of another framework?
00:30:14.430
We find ourselves thinking that while dry-web acts as another framework, the goal is to focus on our domain model first. We ought to structure our code in a way that allows for clear separation.
00:30:39.520
For someone who’s new to dry-rb, can these gems be incorporated into an existing Rails application?
00:30:54.640
Absolutely! You can gradually introduce dry-rb functionalities into any existing application. It’s designed to work independently of Rails while complementing the existing infrastructure.
00:31:17.100
This flexibility allows you to upgrade your applications sooner without requiring a full rewrite.
00:31:49.670
Thank you, everyone! If there are no more questions, we appreciate your attention!
