Talks
Merb, All you need, nil you don't
00:00:15.240
So today we're going to talk a little bit about Merb and why it may be cool or why it may not be cool. My name is Ezra Zygmuntowicz, and I work at Engine Yard. We build clusters and stuff.
00:00:22.199
We're going to go through a couple of the guiding principles of Merb development, and then we'll take a look at some of the more advanced features. One of the main goals here is to prefer simplicity over magic as much as possible. Simple code runs better; it's easier for other people to understand. When you're building framework-style code that lots of other people are going to use, you really need to keep this in mind.
00:00:39.840
None of the special tricks or complex methods belong in framework code. It's fine for your own application because you're in control of it, but if you're going to make this big ball of code and hand it to other people, you need to make it pretty sane. When in doubt, benchmark and profile your code. Merb is fast, and it's fast because I haven't made random assumptions about how Ruby works. I've actually benchmarked and profiled, and you can't really get performance numbers better without doing this. You're almost always going to be wrong if you're guessing at what's taking the time in your program.
00:01:06.640
Another important issue is knowing your runtime and how it acts. Ruby is a complex beast, and there are all kinds of little corners of the language that you might not expect. I have a little folder on my Mac where I keep various idioms you can do in Ruby in different ways, with benchmarks for each of them. You can find some simple benchmarks in the Merb core Git repository as well.
00:01:37.880
This is my big motto: no code is faster than no code. If you just have as little code as possible, it's going to execute faster than some big, cumbersome monster that is a monument to personal cleverness. So let's talk about why Merb is appealing. We're all hackers here. Merb is all about clarity; you should be able to look into your framework and figure things out and hack it to bend it to your will. In my opinion, a framework should not be a black box that people never explore because it's intimidating.
00:02:14.239
A significant factor is to not leave any broken windows. As a project gains momentum and more people contribute, if you start to leave little annoying bugs or parts of the code base that could be cleaner or need better testing, that builds up. Anyone else working on the project will see that and think, 'Oh, well, it's kind of janky over here, so I don't mind if my code is janky too.' Merb is all about web services, and this was originally what it was written to deal with — file upload services and little REST services.
00:03:16.519
A lot of people using Merb in production right now have a Rails app for their main UI tier and Merb services on the backend that handle specific tasks. It's efficient at making small memory footprint servlets, so you can have many of them performing different functions.
00:03:52.239
This example here shows how Merb provides an API. It approaches responding to formats differently than Rails does. In your controller, you can declare that a post controller provides JSON, YAML, and XML, while all controllers provide HTML by default. When a request comes in, we check the MIME types registered at the top. Assume the accept headers are set to application/x-yaml or text/yaml; we can display the post accordingly. This is a clean way to declare that a controller provides different formats and makes it easy based on the accept header to automatically call the respective method.
00:05:01.840
Another cool feature of Merb is that we use parse tree in Ruby on server boot. We walk through all the controller classes and any methods they mix in for modules, looking at their arguments. This way, rather than just using a generic params hash, we specifically find out the parameters for each action. Merb memorizes these details so when we dispatch for a post show action, it already knows what parameters it needs.
00:05:31.280
So, instead of merely looking at params ID or params user, we have direct access to the parameters our actions are defined with. This tends to align more closely with Ruby principles in my opinion. Additionally, the return value from your actions is what gets sent to the client. There is no automatic rendering; you will always need to call render, display, or you can return a string directly, which will be sent back to the client.
00:06:09.080
You can also return an IO handle to stream directly to the client, or return a proc object that takes a response and allows direct writing to it. There's a lot of advanced streaming technology in Merb. A simple example is the ability to run a Merb app and watch it tail its own logs. The render chunked function uses chunked transfer encoding, keeping the connection to the client open to send little chunks down the stream.
00:06:57.919
You could have this running for AJAX progress uploads or stream chunks of JavaScript to an iframe to execute when they hit. Also, a lot of people use S3 for file storage, and if you have a private file on S3 that requires authentication before access, you can have your Merb application handle the authentication first. The stream file method allows the app to authenticate the user before initiating the download. After authentication, you can specify the file name, type, and content length, yielding a response that allows you to stream each chunk from S3 directly to the client.
00:08:24.560
This means that the Merb app acts as a proxy between the client and S3, passing requests through directly rather than having to fully download the file to the server before sending it to the client. This results in a much faster experience for users requiring streaming functionality.
00:08:53.600
Merb is built on Rack, a web server extraction library that simplifies web requests down into a Proc or any Ruby object with a call method that takes an environment hash. The environment hash contains standard CGI headers and the body IO stream if there's a post body. This abstraction allows the same code to run on any of these web servers. An example of a Rack application can be created simply by having anything respond to call, taking an environment argument and returning a tuple with the response status, headers, and body.
00:09:27.880
In all Merb applications, you'll find a config.ru file that runs the Merb Rack application. This file sets up a Rack application for the Merb code to dispatch incoming requests.
00:10:16.000
This makes it powerful to expose Rack in your Merb app. For example, if you have an API for XML, RSS feeds, or JSON, you can set up an API handler in front of the Merb Rack application. When a request comes in, the handler initializes and checks the path to determine if it matches the API endpoint. If it does, it responds directly without going through the main app.
00:10:39.480
Additionally, you can set up a Rack Cascade, which is an array of Rack applications. The first one that returns a non-404 will be sent to the client. This allows you to handle multiple applications in the same process. Furthermore, there are various middleware available for logging or managing profiling in Merb.
00:11:45.920
I want to shift our focus to web servers. Mongrel has been the standard server for running Rails and Merb applications. It's a solid, stable choice as a threaded server. Each request spawns a new thread, which handles the request independently. However, new event-driven servers like Thin and EventMachine are emerging that utilize non-blocking approaches. These servers can be much faster but may struggle with long actions or file uploads that block the event loop.
00:13:13.280
In working with the authors of Thin and EventMachine, we've implemented a deferred method for long-running actions in this context. It allows spawning a thread if a long request is detected, enhancing the performance while maintaining the benefits of an event-driven server.
00:13:49.760
Merb offers deferred callables using the render deferred method, which allows you to set up a proc that will be executed later. This separation means the server can process other requests efficiently while still handling lengthy actions in the background. There's also a render then call method that allows you to give a response to the client immediately and process further actions afterward.
00:14:57.600
The routing system in Merb is powerful, capable of handling most of the features of Rails routing, including nested resources and namespaces.
00:15:49.080
For example, if a request matches a specific user-agent, you can direct that request to a unique controller. Moreover, Merb supports deferred routes that enable executing arbitrary Ruby code during the routing process, allowing for dynamic routing based on conditions like looking up a subdomain in the database.
00:16:27.080
If you're interested in contributing, you can find the Merb core on GitHub and Lighthouse. We have an IRC channel, Google Group, and a wiki that is written in Merb and is collecting helpful resources. Additionally, Merb was heavily inspired by Rails, which has led to ongoing efforts for re-architecturing.
00:17:51.480
In particular, I've focused on integrating some of the coolest features of Merb back into Rails, especially the Rack machinery, and standardizing the command-line tools for deployment.
00:18:24.320
I've restructured the parameter parsing in Rails to remove dependencies on outdated libraries like CGI.rb. This change is crucial for improving performance and streamlining how Rails handles requests.
00:19:24.520
There are several concepts surrounding thread safety that we'll address, especially around dependency loading. Creating a preload mode that loads all necessary components before a request is dispatched will help mitigate issues that can arise.
00:20:17.440
Now, I am open to questions about these changes and any other aspects you'd like to discuss regarding Merb or Rails performance.
00:21:53.520
[Some audience Q&A cuts here. Questions include discussions around caching with memcached, S3 streaming, and expectations around performance during production runs. Each question raises specific points about memory handling, thread performance, and project experiences with long-running processes that neither libraries manage reliably.]
00:22:25.960
To summarize, I'd like Rail's threading to mirror Merb's capabilities, ensuring every request maintains independence without significant overhead. Thanks for your time, and feel free to reach out for further questions or discussions.
