Merb, All you need, nil you don't

Talks

Ezra Zygmuntowicz

Merb, All you need, nil you don't

by Ezra Zygmuntowicz

In the talk titled "Merb, All you need, nil you don't" by Ezra Zygmuntowicz at GoRuCo 2008, the speaker explores the features and guiding principles of the Merb web framework, contrasting it with Rails while emphasizing performance and best coding practices.

The key points discussed include:
- Simplicity over Complexity: Zygmuntowicz stresses the importance of writing simple code free from unnecessary complexity, especially in framework development, as simpler code is easier for others to understand and maintain.

- Performance Focus: Merb is designed with performance in mind. The speaker advocates for benchmarking and profiling code rather than guessing where bottlenecks may occur.
- Clarity and Hackability: Merb champions transparency, allowing developers to explore the framework easily without intimidation. It is tailored for web services, supporting functionalities such as small memory footprint servlets and easy file uploads.

- Advanced Features: The talk details Merb's approach to handling different response formats through controller declarations, leveraging Ruby's parse tree for enhanced parameter management, and using streaming technology for efficient file handling.
- Rack Integration: Merb is built on Rack, which enhances its compatibility with various web servers and allows for versatile API handling, middleware integration, and request routing based on conditions.

- Threading and Performance: Zygmuntowicz discusses using Mongrel as a server while comparing it to newer event-driven servers like Thin. He explains how deferred callables in Merb allow efficient handling of long-running requests to maximize server performance.

- Community and Contribution: The open-source nature of Merb invites contributions through platforms such as GitHub and emphasizes community support via IRC and Google Groups.

The speaker concludes with a call for contributions and a desire to integrate Merb's modern features back into Rails, ensuring performance improvements and thread safety. Zygmuntowicz expresses optimism about making Rails threading functions more independent to handle requests seamlessly. The talk ends with interactions from the audience, addressing questions on caching, streaming, and best practices in production environments.

In summary, the video provides an informative overview of Merb, highlighting its advantages, architecture, and community focus while promoting best practices in code simplicity, performance, and usability.

00:00:15.240 So today we're going to talk a little bit about Merb and why it may be cool or why it may not be cool. My name is Ezra Zygmuntowicz, and I work at Engine Yard. We build clusters and stuff.

00:00:22.199 We're going to go through a couple of the guiding principles of Merb development, and then we'll take a look at some of the more advanced features. One of the main goals here is to prefer simplicity over magic as much as possible. Simple code runs better; it's easier for other people to understand. When you're building framework-style code that lots of other people are going to use, you really need to keep this in mind.

00:00:39.840 None of the special tricks or complex methods belong in framework code. It's fine for your own application because you're in control of it, but if you're going to make this big ball of code and hand it to other people, you need to make it pretty sane. When in doubt, benchmark and profile your code. Merb is fast, and it's fast because I haven't made random assumptions about how Ruby works. I've actually benchmarked and profiled, and you can't really get performance numbers better without doing this. You're almost always going to be wrong if you're guessing at what's taking the time in your program.

00:01:06.640 Another important issue is knowing your runtime and how it acts. Ruby is a complex beast, and there are all kinds of little corners of the language that you might not expect. I have a little folder on my Mac where I keep various idioms you can do in Ruby in different ways, with benchmarks for each of them. You can find some simple benchmarks in the Merb core Git repository as well.

00:01:37.880 This is my big motto: no code is faster than no code. If you just have as little code as possible, it's going to execute faster than some big, cumbersome monster that is a monument to personal cleverness. So let's talk about why Merb is appealing. We're all hackers here. Merb is all about clarity; you should be able to look into your framework and figure things out and hack it to bend it to your will. In my opinion, a framework should not be a black box that people never explore because it's intimidating.

00:02:14.239 A significant factor is to not leave any broken windows. As a project gains momentum and more people contribute, if you start to leave little annoying bugs or parts of the code base that could be cleaner or need better testing, that builds up. Anyone else working on the project will see that and think, 'Oh, well, it's kind of janky over here, so I don't mind if my code is janky too.' Merb is all about web services, and this was originally what it was written to deal with — file upload services and little REST services.

00:03:16.519 A lot of people using Merb in production right now have a Rails app for their main UI tier and Merb services on the backend that handle specific tasks. It's efficient at making small memory footprint servlets, so you can have many of them performing different functions.

00:03:52.239 This example here shows how Merb provides an API. It approaches responding to formats differently than Rails does. In your controller, you can declare that a post controller provides JSON, YAML, and XML, while all controllers provide HTML by default. When a request comes in, we check the MIME types registered at the top. Assume the accept headers are set to application/x-yaml or text/yaml; we can display the post accordingly. This is a clean way to declare that a controller provides different formats and makes it easy based on the accept header to automatically call the respective method.

00:05:01.840 Another cool feature of Merb is that we use parse tree in Ruby on server boot. We walk through all the controller classes and any methods they mix in for modules, looking at their arguments. This way, rather than just using a generic params hash, we specifically find out the parameters for each action. Merb memorizes these details so when we dispatch for a post show action, it already knows what parameters it needs.

00:05:31.280 So, instead of merely looking at params ID or params user, we have direct access to the parameters our actions are defined with. This tends to align more closely with Ruby principles in my opinion. Additionally, the return value from your actions is what gets sent to the client. There is no automatic rendering; you will always need to call render, display, or you can return a string directly, which will be sent back to the client.

00:06:09.080 You can also return an IO handle to stream directly to the client, or return a proc object that takes a response and allows direct writing to it. There's a lot of advanced streaming technology in Merb. A simple example is the ability to run a Merb app and watch it tail its own logs. The render chunked function uses chunked transfer encoding, keeping the connection to the client open to send little chunks down the stream.

00:06:57.919 You could have this running for AJAX progress uploads or stream chunks of JavaScript to an iframe to execute when they hit. Also, a lot of people use S3 for file storage, and if you have a private file on S3 that requires authentication before access, you can have your Merb application handle the authentication first. The stream file method allows the app to authenticate the user before initiating the download. After authentication, you can specify the file name, type, and content length, yielding a response that allows you to stream each chunk from S3 directly to the client.

00:08:24.560 This means that the Merb app acts as a proxy between the client and S3, passing requests through directly rather than having to fully download the file to the server before sending it to the client. This results in a much faster experience for users requiring streaming functionality.

00:08:53.600 Merb is built on Rack, a web server extraction library that simplifies web requests down into a Proc or any Ruby object with a call method that takes an environment hash. The environment hash contains standard CGI headers and the body IO stream if there's a post body. This abstraction allows the same code to run on any of these web servers. An example of a Rack application can be created simply by having anything respond to call, taking an environment argument and returning a tuple with the response status, headers, and body.

00:09:27.880 In all Merb applications, you'll find a config.ru file that runs the Merb Rack application. This file sets up a Rack application for the Merb code to dispatch incoming requests.

00:10:16.000 This makes it powerful to expose Rack in your Merb app. For example, if you have an API for XML, RSS feeds, or JSON, you can set up an API handler in front of the Merb Rack application. When a request comes in, the handler initializes and checks the path to determine if it matches the API endpoint. If it does, it responds directly without going through the main app.

00:10:39.480 Additionally, you can set up a Rack Cascade, which is an array of Rack applications. The first one that returns a non-404 will be sent to the client. This allows you to handle multiple applications in the same process. Furthermore, there are various middleware available for logging or managing profiling in Merb.

00:11:45.920 I want to shift our focus to web servers. Mongrel has been the standard server for running Rails and Merb applications. It's a solid, stable choice as a threaded server. Each request spawns a new thread, which handles the request independently. However, new event-driven servers like Thin and EventMachine are emerging that utilize non-blocking approaches. These servers can be much faster but may struggle with long actions or file uploads that block the event loop.

00:13:13.280 In working with the authors of Thin and EventMachine, we've implemented a deferred method for long-running actions in this context. It allows spawning a thread if a long request is detected, enhancing the performance while maintaining the benefits of an event-driven server.

00:13:49.760 Merb offers deferred callables using the render deferred method, which allows you to set up a proc that will be executed later. This separation means the server can process other requests efficiently while still handling lengthy actions in the background. There's also a render then call method that allows you to give a response to the client immediately and process further actions afterward.

00:14:57.600 The routing system in Merb is powerful, capable of handling most of the features of Rails routing, including nested resources and namespaces.

00:15:49.080 For example, if a request matches a specific user-agent, you can direct that request to a unique controller. Moreover, Merb supports deferred routes that enable executing arbitrary Ruby code during the routing process, allowing for dynamic routing based on conditions like looking up a subdomain in the database.

00:16:27.080 If you're interested in contributing, you can find the Merb core on GitHub and Lighthouse. We have an IRC channel, Google Group, and a wiki that is written in Merb and is collecting helpful resources. Additionally, Merb was heavily inspired by Rails, which has led to ongoing efforts for re-architecturing.

00:17:51.480 In particular, I've focused on integrating some of the coolest features of Merb back into Rails, especially the Rack machinery, and standardizing the command-line tools for deployment.

00:18:24.320 I've restructured the parameter parsing in Rails to remove dependencies on outdated libraries like CGI.rb. This change is crucial for improving performance and streamlining how Rails handles requests.

00:19:24.520 There are several concepts surrounding thread safety that we'll address, especially around dependency loading. Creating a preload mode that loads all necessary components before a request is dispatched will help mitigate issues that can arise.

00:20:17.440 Now, I am open to questions about these changes and any other aspects you'd like to discuss regarding Merb or Rails performance.

00:21:53.520 [Some audience Q&A cuts here. Questions include discussions around caching with memcached, S3 streaming, and expectations around performance during production runs. Each question raises specific points about memory handling, thread performance, and project experiences with long-running processes that neither libraries manage reliably.]

00:22:25.960 To summarize, I'd like Rail's threading to mirror Merb's capabilities, ensuring every request maintains independence without significant overhead. Thanks for your time, and feel free to reach out for further questions or discussions.

GoRuCo 2008