00:00:07
Please welcome Erica with your talk.
00:00:10
It’s nice to see you here. I hope that you had something nice for lunch, but maybe not too nice, so you don't fall asleep.
00:00:17
I’m going to start by asking you a couple of questions. How many of you use Rails today?
00:00:26
I’m trying to decide how harsh I’m going to be. No, just kidding. I like Rails.
00:00:35
So, how many of you know Hanami or have worked with Hanami? Yeah, pretty many. That’s nice!
00:00:45
Today, I’m going to talk about web development in Ruby at 84 Codes, the company I work for. I'll start off by explaining what we do and how that affects our system architecture and development principles.
00:00:59
Then, I will give you a short introduction to Sinatra, which is the web framework that we use, and also introduce Rails and Hanami, just to get everyone up to speed.
00:01:10
At last, I will go through our web application setup and explain how we manage with Sinatra and other tools, but foremost, what we can improve based on what I have learned.
00:01:26
Okay, but first an introduction about me. My name is Erica, and I'm based in Stockholm. I usually go to the gym, swimming, running, skiing, or biking – and even more biking.
00:01:42
I work as a developer at 84 Codes, where I have been for almost four years now, but I have only worked full-time for about two years, so please be kind.
00:02:02
Currently, I work in our observability team, which is responsible for monitoring our systems as well as our customer servers. This means managing logs, metrics, alarms, and much more.
00:02:14
I’m also very interested in developer experience, and that’s what brought me into this topic today. So enough about me, let’s dig into 84 Codes.
00:02:29
84 Codes was founded in 2012 and we are now around 40 employees, with roughly half of us being developers.
00:02:42
We are mainly known for Cloud MQ or message broker as a service. We maintain the largest fleet of RabbitMQ and LMQ services worldwide, which consists of over 15,000 servers.
00:02:54
I want to ask you one more question, so how many of you are familiar with and have worked with RabbitMQ?
00:03:10
More than I thought! That’s nice. RabbitMQ is one of the most used message brokers, and we have provided RabbitMQ as a service for quite some time now.
00:03:22
LMQ is our own built message broker released in 2022, and both LMQ and RabbitMQ follow the AMQP protocol, which is the Advanced Message Queuing Protocol.
00:03:38
I’m going to explain shortly what a message broker is, how it works, and what it’s used for, because I think it’s important to understand how we develop web applications.
00:03:50
Think about a system built of multiple services. Let's say a web shop, where one service is responsible for the web shop content, one service handles the cart and checkout, and another service is responsible for the payments.
00:04:01
These services need to communicate with each other and the message broker allows them to do that asynchronously with the help of message queues.
00:04:19
For example, a customer of the web shop adds something to their shopping cart and proceeds to checkout. The checkout service, called the producer, sends a message to the message broker.
00:04:37
The message broker puts the message in the correct queue defined by the sending service. The producer and the consuming service, in this case, the payment service can then subscribe to the queues they are interested in.
00:04:55
The consuming service can then consume the message once it has arrived in the queue and start to process the payment.
00:05:03
When the payment is done, the service can send a confirmation to another service that’s responsible for notifying the customer about the payment.
00:05:19
This is a simplified explanation of what a message broker does. In reality, there’s a bit more to it and if you are interested in learning more, I highly recommend the CloudAMQP website or the free RabbitMQ course on training.cloudamqp.com.
00:05:41
Now, let’s move on to how Ruby is used at 84 Codes and also touch on some other languages we work with and the kind of projects we maintain.
00:05:51
We have approximately 20 applications written in Ruby. These are our main applications, but we also have around 15 applications written in other languages, which are used for tasks like notifications, metrics, logs, alarms, and integrations.
00:06:05
Some of the other languages we code in are JavaScript, Clojure, and Crystal. Crystal is a language with syntax very similar to Ruby but is faster and more efficient due to being a compiled language.
00:06:22
We use Crystal for applications where efficiency is important. For example, we use it in open-source message broker LMQ.
00:06:35
We also maintain some open source software like RabbitMQ, along with proxy servers, AMQP clients, a Terraform provider, and more.
00:06:50
Another reason for using all these languages is that the goal at 84 Codes has always been to allow developers to do what they think is fun.
00:07:05
While Ruby is our main language, developers are always encouraged to try out new languages when they want or when it fits the use case.
00:07:21
This is a simplified version of the architecture of the main part of the CloudAMQP system. We have a main service which handles interactions with customers.
00:07:41
This service sends messages through LMQ to the job service, which handles communication with our cloud providers such as AWS, Azure, Google Cloud, and Digital Ocean.
00:07:55
It also installs and configures RabbitMQ and LMQ and manages everything else regarding the actual servers.
00:08:09
Then we have a listener service that ensures that data gets updated in the database. Other services subscribe to messages in LMQ as well.
00:08:24
This is one of the main reasons why using a message broker is great for us, because other services can listen to the queues they’re interested in and get the messages they need to do their job.
00:08:42
This also allows us to use services in different types of languages without any hassle.
00:09:01
Our main web applications are built in Ruby and with Sinatra. So why is that? When 84 Codes was founded, Rails was not as widely used as it is today.
00:09:24
The reason for choosing Sinatra was mostly due to the desire to start small. The service was built as API-only from the start, which didn’t match Rails back then since it was more for full-fledged web applications.
00:09:40
Only later did Rails add their API-only option, making it easier to use for API applications. Sinatra matched our criteria better, and since then we have just continued to work with it.
00:09:56
Although we have started to look into alternatives, investigating Rails and Hanami to find out what we are missing by not using other frameworks.
00:10:07
The motivation behind looking into alternatives is primarily to get more support from the framework, but also to make it easier to hire new developers with good experience in the framework we use.
00:10:19
So what is Rails? It’s a web framework for Ruby that provides routing, request handling, and view management.
00:10:30
Sinatra requires a Rack-compatible web server, which we use Puma, and I will talk more about the Sinatra web stack, but first I want to briefly introduce the two other frameworks.
00:10:41
Let's start with Rails, which is by far the most used Ruby web framework today. Rails is optimized for happiness and probably works well for many developers.
00:10:57
However, Rails is a very opinionated framework, and the philosophy is convention over configuration. It was built to make web development easier and faster by allowing developers to write much less boilerplate code.
00:11:10
As long as you follow Rails conventions, it's very easy to set up a Rails application. However, this also means that things happen very magically, and it can be a bit confusing to understand how it works.
00:11:25
Hanami is also a web framework but was built to be a very lightweight, modular, and performant framework compared to Rails’ more monstrous approach.
00:11:38
Now, back to Sinatra and our web stack. One of the things we miss in Sinatra is that it doesn't enforce a specific structure.
00:11:53
I think the only thing Sinatra expects by default is that views are placed in a views folder, but that path can be changed as well, leaving everything up to the developer.
00:12:08
If you create an application without a defined architecture from the beginning, it may become a real mess.
00:12:30
Here is how our architecture looks. It’s inspired by Rails and the Model-View-Controller (MVC) pattern.
00:12:40
Controllers receive requests and forward them to the correct actions, and those actions interact with the model to update the views.
00:12:56
However, the layer between the database and the controller is where things have gone a bit wrong for us due to not having a forced structure.
00:13:12
In some places, we have controllers communicating directly with the database, while others communicate with a model which then interacts with the database.
00:13:30
Some controllers communicate with a service which then communicates either with the model or directly with the database.
00:13:39
This creates confusion and misunderstandings when the patterns differ in the code, leading to increased lead time for developing changes.
00:13:58
We are trying to unify this by ensuring that the controller always communicates with a service, then a model, and finally with the database.
00:14:17
This approach aligns with the MVC structure, but Rails is also quite flexible, and it’s not unusual to find Rails applications with more abstractions.
00:14:45
For example, we try to avoid having an intermediary service between the controllers and the models, which can complicate the architecture.
00:15:05
Since Rails applications are often monoliths, they can grow really complex without those abstractions.
00:15:18
Hanami also follows the MVC pattern but with much more abstraction. In Hanami, controllers are not present and requests are processed directly by actions.
00:15:36
Each action is a self-contained class, and by default, Rails renders its equivalent views. Views in Hanami are abstracted into parts and templates.
00:15:54
Parts include view logic, and templates represent the actual HTML structure. The default template engine used is ERB in Hanami, and we also use HAML.
00:16:05
The model portion of a Hanami application is abstracted into repositories, relations, and entities. Repository objects provide abstractions over the ORM.
00:16:28
This allows actions to communicate with ordinary Ruby objects. I like the idea of more abstractions, but I don’t have enough knowledge to have a strict opinion about it.
00:16:47
Another thing Rails and Hanami enforce is structure. If the application follows the conventions that the site files expect, you don't have to worry about requires.
00:17:01
This significantly improves the usability of the application, allowing you to focus on development.
00:17:14
When changing files in the application, Sitework automatically loads your classes and modules, providing a more pleasant developer environment.
00:17:32
You don’t have to restart your app when making changes to try them out, and in production, it allows you to eager load the application.
00:17:47
In our development environment, we use Reloader, which is a Ruby gem that can restart any process when files change.
00:18:09
In our case, it restarts Puma. This approach will probably be a bit slower than just reloading classes like Sitework does, but it works for us.
00:18:29
Now I'm going to talk about something that Sinatra actually provides: routing. This is an example of how routing can be done with Sinatra.
00:18:47
The routes are defined in the config route clause, and each controller includes its own route. However, this structure makes it a bit hard to get an overview of all the endpoints.
00:19:01
Rails, on the other hand, has all routes configured in a single routes file, which makes it easy to get an overview of the application.
00:19:19
This allows you to quickly see which endpoints the application has, making it easier to manage.
00:19:38
One thing nice in both Rails and Hanami is the command to list all routes in your application, along with the controller and action it points to.
00:19:58
This feature is very handy for quickly obtaining an overview of the application’s routes.
00:20:10
Sinatra doesn’t provide an ORM for database integration, so we use SQL, a well-used and maintained ORM that fits our purpose well.
00:20:36
The maintainer, JY Evans, does a great job of maintaining SQL, where issues are never open for long and the documentation is comprehensive.
00:20:53
However, it’s always showing Active Record examples when I try to use GitHub Copilot and many solutions on Stack Overflow are Active Record-based.
00:21:07
In our case, models in SQL are defined slightly differently. Instead of using named relations like 'has_many' or 'belongs_to', we define relations using regular one-to-many or many-to-one structures.
00:21:23
We have implemented our own migration tasks that take versions as arguments, migrating to the latest if no specific version is passed.
00:21:39
Additionally, we have implemented a seeds task and an annotations task, where annotations in SQL add table schema to model files for better clarity.
00:22:01
This makes it very easy to understand what you are dealing with in a model. As for migrations, Rails and Hanami provide easy commands to create apps and generate boilerplate code.
00:22:27
In both Rails and Hanami, you can set specific flags when creating a new app, such as avoiding certain components that you don't need.
00:22:45
For example, in Rails, I would skip Active Job since we use microservices and usually schedule tasks in another service.
00:23:04
Similarly, we don’t require Action Cable for web socket support or Action Mailer since we have our own mailer service.
00:23:20
In Rails, you also have the option to create an app with minimal flags, and for Hanami, you can skip assets when creating an API.
00:23:34
When generating resources in Rails, you can create controllers and models, while in Hanami, you generate actions, parts, and views.
00:24:01
In Rails, you also have the occasion to use scaffold, which provides you a controller, model, view, routes, and some test files.
00:24:15
However, you still need to write and refine your models afterwards.
00:24:33
In Sinatra, you generally end up doing more copy-paste work, but all solutions work as long as they meet your needs.
00:24:47
Another useful command in Rails and Hanami is the ability to interact with your application through the command line using IRB. Hanami also allows using Pry for this purpose.
00:25:10
We have implemented this feature ourselves with just a few lines of code.
00:25:28
As a last part, I want to talk about the testing frameworks we use. We use MiniTest, but in a spec style that resembles RSpec.
00:25:46
This style helps organize the tests in a more behavioral and contextual manner, using describe blocks to group related tests. The expect syntax makes conditions readable and descriptive.
00:26:23
Rails uses ActiveSupport TestCase with MiniTest, while Hanami uses a framework called RSpec.
00:26:39
In conclusion, after our investigation into different frameworks, we see that the biggest reason for not continuing with Sinatra is its relatively small and inactive community.
00:27:01
Transitioning our applications to Rails or Hanami would require substantial resources, but we may try it out and see what works best.
00:27:21
We have implemented most of what we think is needed ourselves or with the help of Gems, such as mailer services and rake tasks for migrations.
00:27:38
One thing I really want to experiment with is Sitework; it would be interesting to see if we could implement it in our applications.
00:27:59
So, that was it for me today. I have learned so much while writing this talk, and I truly appreciate that you have listened.
00:28:15
I hope that you have learned something as well. Thank you!