00:00:14.710
Good morning everyone, and welcome to my talk titled 'More Than Syntax.' My name is Patrick Huesler.
00:00:24.320
You can find me on Twitter as @fueler. I've been told by a trainer that you should not give out your name on stage, but I think that's just rude. So I decided to introduce myself.
00:00:37.399
Originally, 'More Than Syntax' was the title of my talk, but on the way here, I changed my mind. Like a good startup, I pivoted my talk.
00:00:49.280
I changed the title to 'Web APIs with Erlang: A Ruby Developer's Point of View.' In this talk, I will discuss how you can write web services in Erlang and how to integrate your Ruby tools, automations, and deployments within your environment.
00:01:08.780
Over the last three months, my focus has been on achieving this integration.
00:01:21.170
Also, on a very important note from yesterday, Terence Lee is not a liar. He mentioned that he is looking forward to live coding in my talk, but that's not going to happen.
00:01:32.570
The reason is that I tend to struggle with coding while framing coherent sentences, and I would prefer to spare you all the embarrassment that could lead to.
00:01:45.469
So there isn't a liar; in fact, Terence is awesome, especially with his work on the Rails group. My topic is cool for Hawaii, I guess.
00:01:52.729
I grew up in Switzerland, in the mountains. I grew up around snow, so snowboarding, skiing, and ice hockey— you name it, I did it. There was even one incident where I went to kindergarten with my skis on.
00:02:04.440
I'm fairly proficient in snowboarding and understand how it works. This brings me to the topic of surfing. I have this slide so I can wear shorts on stage as a prop without feeling embarrassed.
00:02:19.209
Additionally, I wanted to learn surfing. I thought it would be easy, given my experience snowboarding.
00:02:31.110
I figured it would be a piece of cake, so I read a book that detailed every aspect of the sport. I thought, 'Well, that's great, I can do this!'
00:02:51.250
However, one day I went out surfing, wearing my board shorts. In Europe, most of the time you need a wetsuit because the water is too cold.
00:03:07.690
So, I did the surf check, looking for waves, currents, and where the lineup was, then I attempted to paddle out.
00:03:14.829
I fell off the board immediately. I didn't know how wobbly those boards could be, so I struggled to paddle.
00:03:26.290
The next day, at the surf spot, I tried again to get out, and I managed to work around my difficulties.
00:03:32.320
I planned to catch my first wave and felt confident that it would be as easy as snowboarding.
00:03:44.410
But this did not happen, and I ended up falling repeatedly for the whole afternoon.
00:03:51.450
The lesson here is that sometimes, just reading a book and banking on your previous knowledge doesn't translate well to a new skill.
00:03:57.720
This brings me to Erlang, a similar story to my experience with Ruby.
00:04:10.340
Just a quick check: who here has looked into Erlang?
00:04:19.230
And who here has written production code in Erlang?
00:04:28.520
So I consider myself an Erlang beginner; I've only been using it productively for a couple of months. I started learning it about four years ago, but never went beyond a simple chat server or 'Hello, World!' application.
00:04:42.180
This means that my perspective comes from that of a beginner, so there might be things in my talk that experienced Erlang developers might disagree with.
00:04:55.290
I will not cover an introduction to Erlang here, as others have done that much better.
00:05:02.700
My focus will be on the concepts and the tools I needed to combine my Rails and Ruby mindset with Erlang in a more original way, rather than trying to shoehorn Ruby practices into Erlang.
00:05:15.240
This will involve a lot of content and code, but I won't go into every single detail. I have everything uploaded to a GitHub repository and I deployed the application on Heroku.
00:05:27.510
To make sure we're on the same page, a quick Erlang refresher: Erlang is a functional language based on mathematical principles. Everything is a mathematical function, and it strives to avoid state and mutable data.
00:05:44.400
This approach is particularly useful for the kinds of things you build in Erlang.
00:05:50.130
Erlang is a compiled language that builds bytecode known as BEAM files and executes it via a virtual machine. This VM handles everything, including connections between nodes and clusters.
00:06:03.390
Additionally, Erlang OTP provides a set of common practices, patterns, and abstractions to aid in building robust applications in Erlang, like generic servers, state machines, and event handling.
00:06:19.020
Now, why Erlang? First and foremost, Erlang is highly concurrent—it was specifically built for that.
00:06:38.200
Managing CPU and I/O concurrency is straightforward in Erlang.
00:06:43.720
You don't need to worry about non-blocking I/O or event machines since they are inherently built into the language.
00:06:50.480
Erlang uses an actor model which employs message passing, ensuring that there is no shared data that could be corrupted. This avoids the issues of data corruption and the need for locks when accessing shared data.
00:07:08.200
You can see analogous behavior in the Ruby world with frameworks like Celluloid, which provides an actor-based approach, and it's particularly effective in JRuby and Rubinius.
00:07:21.250
Erlang is also designed for distributed systems, making networking almost trivial.
00:07:28.420
You can communicate with other machines and build service-oriented architectures seamlessly.
00:07:35.620
If you look at database systems like CouchDB or Riak, many are written in Erlang.
00:07:43.450
Another core attribute of Erlang is fault tolerance. Erlang systems are designed to run continuously and expect failure.
00:07:51.220
One of the main strategies is to let processes crash, allowing the infrastructure to restart them automatically when things go wrong.
00:08:04.870
This provides great strategies around what to do when something goes wrong and what gets restarted.
00:08:12.310
One of my favorite features is the ability to do live coding; this can be really beneficial when you're in a production environment.
00:08:27.220
For example, when you're running low on time and need to get things done, Erlang allows you to make changes without tearing everything down.
00:08:38.340
Erlang is a compiled language, yet it allows for hot code swapping.
00:08:43.570
This means that you don't have to shut down the entire virtual machine or system to apply updates.
00:08:52.080
It provides infrastructure for reloading code and modifying it on-the-fly.
00:09:00.150
You can attach remotely to every node in your Erlang cluster and operate similarly to an IRB within the context of your running virtual machine.
00:09:09.620
At MOG, the company I work for, we started using Erlang because we develop games on Facebook and iOS, such as Monster World.
00:09:29.360
A critical aspect of our game is sharing data among friends, where players can interact and help each other.
00:09:43.160
We face technical challenges with our backend due to high user volume—about 20 million users each month, with roughly 4 million users daily.
00:10:05.060
From our data, we observed peak traffic reaching 6,500 requests per second, which fluctuates based on user activity.
00:10:21.130
The real challenge lies in our database; it becomes a bottleneck, especially when handling write requests.
00:10:39.560
We've tried common solutions such as caching and adding read replicas, but these did not solve the writing issues.
00:10:54.300
Eventually, one of our engineers developed a new architecture where the user session information is held in memory instead of on the database.
00:11:06.469
The concept is to eliminate the database pain point altogether: when a player logs in, we create a worker to control their session, loading and saving the data from the cloud in the background.
00:11:29.500
The thought was if we lose a few minutes of data, it's usually not critical in games like this.
00:11:46.410
User sessions exist as separate processes in Erlang, allowing us to run many sessions efficiently on a single machine.
00:11:55.680
The Erlang VM manages these processes, and if one process encounters an error, it won't affect the others.
00:12:10.480
In Erlang, processes are lightweight, making it possible to manage a vast number of them concurrently on a single machine.
00:12:23.930
Next, I want to discuss environments and logging, similar to what you might be familiar with from Ruby, Rails, and Sinatra.
00:12:40.190
The first tool I often use for development is called Rebar, which is a blend of your Rails commands and RubyGems.
00:12:54.260
To start, you create a new application, which generates the necessary skeleton for your code.
00:13:06.190
Erlang has an inherent concept of applications, which allows you to fragment them into manageable parts or services.
00:13:16.990
Next, you will want to set up your dependencies.
00:13:30.350
For example, to access a MySQL database or utilize some specific functionality, you'll need to specify these dependencies.
00:13:44.400
Rebar automatically manages this for you and provides commands to fetch and compile the necessary dependencies.
00:13:55.230
The configuration file for your Erlang application might look something like this.
00:14:09.740
Although it may not be visually appealing, it represents a proper Erlang structure.
00:14:15.569
In this file, you specify the name of the application and its revision.
00:14:22.620
Then you can execute the command to fetch all your dependencies.
00:14:31.640
Next, we need a web server. There are several options in Erlang, including Cowboy, MochiWeb, and others.
00:14:45.540
At our company, we decided to build our own web server because we wanted more control over the request lifecycle.
00:15:02.480
We aimed for deeper insights into the requests to allow for better monitoring. This turned out to be critically important.
00:15:16.650
With minimal code, I even contributed to the creation of a web server in Erlang that is quite fast.
00:15:28.180
I think it's faster than everything I have encountered in Ruby so far.
00:15:40.260
I won’t delve into the details of modules and functions since I'm sure most of you are already familiar with them.
00:15:53.200
In Erlang, you define a callback module that acts like a base class in C, which implements specific functions that get called when requests come in.
00:16:05.410
As you can imagine, having multiple endpoints is vital, so I built a router that behaves somewhat like the routing in Rails.
00:16:20.720
This allows you to define paths and HTTP methods, associating them with specific handlers.
00:16:36.060
The design allows for vivid expression of routes, making it easy to understand request processing.
00:16:46.570
It took me a morning to write this and I felt proud of my accomplishment, though I soon learned from seasoned Erlang developers that I had done it wrong.
00:17:01.470
They advised me to use pattern matching, which is a powerful concept in Erlang.
00:17:16.050
Does anyone here have familiarity with pattern matching? This technique allows you to extract parts of data efficiently.
00:17:26.650
For instance, when defining a function, you can specify expected parameters using a pattern.
00:17:35.450
If Erlang cannot match the given pattern, it will search through the available function definitions to find a suitable match.
00:17:49.560
This gives you flexibility and allows you to write cleaner, more expressive code.
00:18:01.040
Now, let's talk about middleware. Who here likes Rack middleware? I am a big fan of abstraction in that context.
00:18:17.260
It allows you to separate reusable functionalities, such as logging or monitoring, into distinct components.
00:18:32.170
We implemented middleware in our Erlang applications and did so in a way that integrates neatly with the events that happen within the application.
00:18:46.490
There’s a multitude of events you can listen to, which helps you gather metrics and detect issues efficiently.
00:19:04.750
Now let’s shift focus to environments. In any well-rounded application, you typically have different environments for testing, staging, and production.
00:19:16.859
It's crucial that each of these environments has its own configuration.
00:19:31.150
After exploring various solutions in the Erlang ecosystem, I found that passing in configuration parameters via command line was the simplest approach.
00:19:42.480
Erlang supports property list configurations, which are essentially tuples denoting key-value pairs representing your settings.
00:19:55.910
This setup allows you to define various configurations, including application-specific parameters.
00:20:10.340
You can easily adjust logging settings based on user needs, enabling you to gather valuable data about system performance.
00:20:25.560
An important aspect of logging is that it aids in operations by highlighting slow queries and other anomalies.
00:20:38.040
In Erlang, the standard logging approach uses a logger class, which manages log levels and provides comprehensive logging solutions.
00:20:54.080
However, in practice, this generates excessive log data—sometimes you only need concise one-liners.
00:21:06.230
Overall, the complexity of parsing these logs with standard tools can become an obstacle for developers.
00:21:20.850
To address this issue, we explored a tool written by Basho, which offers simpler logging solutions.
00:21:34.930
This solution allows for easier configuration and output that's more manageable to read and parse.
00:21:51.980
Using Logger means you can adjust output to ensure compatibility with tools like syslog, making distributed logging straightforward.
00:22:08.320
A major annoyance in Erlang is managing library dependencies when trying to run applications.
00:22:24.140
Often, errors arise when a sub-dependency cannot be started, which requires a workaround to diagnose.
00:22:39.860
Elements like missing dependencies can create confusion, especially when managing external libraries.
00:22:53.210
While we’re on the subject of dependencies, how is the Erlang code rate? Is it appropriate for everyone?
00:23:06.900
Do keep up the feedback while we proceed with the rest of the content.
00:23:17.570
As we discuss tools for testing, I want to introduce 'Unit,' which is the standard testing framework used in Erlang.
00:23:31.500
I found it effective for basic testing needs, although I ran into an issue with stack traces not being available until more recent releases.
00:23:45.240
With the introduction of version 15, stack traces became accessible, which greatly improved debugging.
00:24:01.550
To create a functional test, you'll follow a typical setup routine similar to most testing frameworks. It's all straightforward.
00:24:17.850
You'll have options like certain assertions that can be called to validate your functionalities.
00:24:31.540
Then, for integration testing of our web services, we played with HTTP requests.
00:24:47.130
One of our team members introduced 'Dtest,' an extension focused on HTTP, which greatly simplifies the integration testing process.
00:25:02.370
With views crafted for the same requests as your application’s, it follows the intuitive structure set in unit tests.
00:25:15.640
Notably, you need to start your application before running HTTP requests against it to get valid responses.
00:25:30.780
Integration testing allows you to confirm your API’s responses without diving into the minutiae of internal components.
00:25:44.930
This framework also supports working with JSON out of the box, making it easier to handle common payloads.
00:25:59.290
Now let’s explore another powerful feature of Erlang: hot code reloading.
00:26:10.340
This feature allows you to save, compile, and start applications without needing to shut the system down.
00:26:23.889
You'll need to embed appropriate modules into your application structure to utilize this effectively.
00:26:41.680
While working with native extensions, however, ensure that you're cautious with code reloading as it does not always carry over seamlessly.
00:27:01.540
One improvement might involve syncing your external libraries to your reload implementation, ensuring stability.
00:27:16.360
Looking over to Heroku, it's an excellent platform I learned about recently.
00:27:32.200
Using Heroku is quite simple; it enables you to deploy applications seamlessly with minimal hassle.
00:27:48.510
Creating your application begins with the standard procedure— establishing a directory and linking it to Git.
00:28:02.480
With the help of buildpacks, you're no longer restricted to one language— you can create apps with various architectures.
00:28:16.970
Once your application is linked with the buildpacks, you're all set up.
00:28:31.120
The process of deploying your Erlang application is straightforward; you only need to provide certain commands.
00:28:48.350
Heroku compiles your project, manages dependencies, and brings everything online effortlessly.
00:29:02.400
You can simply curl your application and verify its functionality, thus confirming if everything was set up correctly.
00:29:15.720
In addition to launching, you have full access to monitor your app's performance.
00:29:28.000
Encouragingly, this is also the stage where you'll find all the codes I've been speaking about.
00:29:42.180
I plan to convert this project into more of an unveiling command, which would facilitate creating new web services.
00:29:58.680
I encourage each of you to venture into creating your first web service in Erlang, as there's much to learn from this powerful language.
00:30:15.040
By gaining insights into new paradigms and concurrency patterns, you can find fresh ways to enhance your Ruby skill set.
00:30:29.080
Overall, Erlang was created with a distinct purpose centered around building reliable, distributed systems that are resilient to failure.
00:30:41.950
Thank you for your attention.
