From Rails to Rack: Making Rails 3 a Better Ruby Citizen

Help us caption & translate this video!

GoRuCo 2009

00:00:19.520 Today, I'm going to talk about making Rails 3 a better Ruby citizen. There are a few things to cover before I start.

00:00:25.160 In the abstract, I focused heavily on Rack. I'm definitely covering Rack in this talk, but I will also go more in-depth about other aspects of how Rails can be a better Ruby citizen, defining what I mean by that and then explaining how we're doing it.

00:00:37.120 Another piece of housekeeping: it's called 'Making Rails 3 a Better Ruby Citizen' and not 'Making Rails 3 a Ruby Citizen.' Rails 2 actually does a reasonably good job of working with other Ruby code. There are a few exceptions—things like JSON or working with other people who use Active Support. So, there are exceptions where Rails 3 doesn't work very well with other libraries, but by and large, Rails is a piece of Ruby code and works well with other pieces of Ruby code.

00:00:48.760 I'm discussing how we can improve it further and do even better than we've already done in the past. So, who am I? I work for a company called Engine Yard. I can keep this introduction short since we've already covered this. Engine Yard sponsored me to work on a project called MB for about a year after I started working for them. MB was basically an experimental fork of Rails, where we took a lot of the ideas from Rails and created something we thought didn't have many of Rails' deficiencies.

00:01:11.439 I worked on that for about a year and towards the end of that year, we discovered that the deficiencies of Rails were absolutely not intractable and, in fact, not conceptually difficult to fix. Once we got everyone on board with actually moving past some of the political and social issues, we determined that it would be rather easy to integrate the ideas that made MB supposedly a separate project into Rails itself.

00:01:35.920 Now, I work full-time on Rails, mostly focusing on refactoring it to create a more modular, integrated experience. Eventually, we'll return to what we were doing with MB and what Rails has been doing historically.

00:01:46.360 I also work alongside the team who develops Engine Solo, which Tom already briefly explained. If you haven't checked it out yet, definitely do so. As Tom pointed out, there is a URL you can visit to enter a contest for some free Solo. Speaking of me, who are you? Who's the audience? The audience is essentially made up of a gradient, with three types of people.

00:02:06.720 There are developers who primarily focus on app development, many of whom have never looked at the internals of Rails and rely almost exclusively on the APIs outlined in 'Agile Web Development on Rails.' They may be doing relatively complex work; they might be navigating complex Ruby tasks. However, for the most part, the people I refer to as app developers don’t delve into the Rails source code or explore plugin or extension APIs. They mostly focus on what Rails offers them as developers.

00:02:34.040 On the other side of the spectrum are those who write extensions or hack on Rails. Personally, I find myself on that end of the scale, and between these two extremes are those who may write applications or focus on extensions but typically end up interacting with the inner workings of Rails over time.

00:02:46.200 When I say 'power user', I mean individuals who are experienced and therefore have their own ideas about utilizing and extending Rails. These are web shops that want to transfer elements from one application to another without having to rebuild them repeatedly, as well as long-term projects that necessitate modifications in how Rails operates over time.

00:03:06.200 The reason I was in the power user category before working on MB and Rails stemmed from a long-term project I started working on before Rails version 1.0. I took that project through to Rails 1.2.3, but along the way, we made numerous tweaks, modifications, and sometimes even forks of the Rails source code. This was due to the fact that our project had a long duration and required us to adapt certain aspects accordingly.

00:03:31.440 We aimed to adhere to the DRY (Don't Repeat Yourself) principle; if Rails already handled a lot of the work, we wouldn't rebuild it ourselves. Instead, we sought to modify Rails to achieve our goals.

00:03:47.680 What do I mean when I say better Ruby citizen? I actually have three different goals in mind. First, I mean that Rails should utilize existing Ruby libraries to avoid reinventing everything itself. For example, it will use TestUnit, which is already established. I'm trying to leverage existing testing infrastructure built on top of Rack. Additionally, Rails 3 is going to start using Ubiquitous, which is a replacement for ERB, allowing us to extend the templating language much better.

00:04:01.560 When I refer to Rails being used as a library for other projects, SproutCore is not built using Rails but utilizes MB. Nonetheless, the idea is that frameworks desiring a lot of the functionality available in Rails or MB could more readily use Rails itself instead of starting from scratch or undergoing extensive hacks.

00:04:16.800 This is about making Rails an accessible tool that can be embedded within other frameworks. An example would be Radiant. Presently, ActiveMerchant utilizes ActiveSupport. Finally, Rails needs to work seamlessly with other languages and libraries.

00:04:31.200 For instance, HAML can be easily integrated into Rails, although it does not fit very well within Rails 2.3 along with JSON or DataMapper. These issues are being resolved in Rails 3. This involves examining specific cases where known issues exist regarding Rails’ compatibility with other libraries and making the necessary improvements.

00:04:50.160 Let's start by discussing how Rails uses existing language features. The most apparent example of this is Rack.

00:05:01.560 So, what exactly is Rack and why is it essential? At its core, Rack serves as a basic interface that states if you have a server, you can communicate with an application through a standard mechanism, while also expecting a standard response from that application. Everyone here who has programmed using frameworks like CGI, Mongrel, Webrick, or IIS knows that prior to Rack, there was a considerable issue where every server would use an interface that resembled CGI but was fraught with inconsistencies. Then, it would expect a very specific type of response unique to the API it was utilizing.

00:05:39.280 In reality, what transpired is that many organizations created tailored implementations specifically for Rails since it was the predominant framework, resulting in other frameworks having to develop their own adaptations as well. Eventually, someone initiated work on a project called Rack, and from the most naive perspective, it appeared to solve the problem of disparate server interfaces by providing a standard means for servers to communicate and respond.

00:06:03.520 When we began working on Merb 1.0, Rack was revolutionary for us because we no longer had to worry about managing numerous adapters supporting different servers. It allowed us to support any server running on top of Rack. We previously maintained separate adapters for different servers in version 0.9.5, so this was a huge breakthrough for us.

00:06:30.560 However, Rack is much more than just a nice common interface; it allows you to build middleware components. If you think about it, Rack is essentially a standard that lays out how to send a request and receive a response, meaning that every element of your machinery within that architecture only needs to receive a request in a particular format with an environment hash and return a response.

00:06:53.120 Middleware simplifies this process as any downstream application does not have to know the details of the request as long as they receive an environment hash. This allows for straightforward processing of requests and responses.

00:07:16.800 Let’s take the example of a path prefix. Previously, if you wanted to implement a path prefix, let’s say for a blog at '/blog/something', you would have to write a considerable amount of code in Rails or Merb just to accommodate that logic. The path prefix middleware now essentially grabs that standard hash's path information and extracts the prefix.

00:07:34.920 So, any downstream Rack machinery sees the URL as if it starts from the root instead of '/blog'. When you generate a response that includes a URL, the middleware simply adds the 'blog' prefix back on. Therefore, downstream applications do not need to concern themselves with the path prefix.

00:07:50.960 A similar concept applies to conditional GET requests; when a request contains an 'Etag' or 'Not Modified' header, it can be short-circuited altogether, meaning we bypass unnecessary processing and return immediately. The static middleware performs similarly; if a request matches a specific file on the server, it returns that file right away instead of traversing through Rails.

00:08:19.440 The middleware thus operates independently, allowing for single-purpose components that do not have to worry about the overall state and can be developed for use with multiple frameworks. Rails has integrated this middleware into Rails 2.3, which illustrates how we can reshape the internal structure of Rails to improve its interoperability.

00:08:43.520 This transformation ultimately allows for a straightforward routing system where the router directs to a Rack request that represents the controller as another Rack endpoint. This modification made it easier to route requests—everything is encapsulated as a Rack application.

00:09:00.960 This means that the controller is just another Rack endpoint, and the router's responsibility is merely to identify which Rack application to send the request to. This transition simplifies the controller structure and functionality.

00:09:19.680 Moving further, Rails 3 aims to enhance this system by ensuring that even when routing requests directed at actions within controllers, we maintain a simple endpoint structure. Furthermore, we minimized magic hashes typically used to identify the actions.

00:09:41.440 Rails 3 now permits the router to point directly at the action. In effect, if you want to access an action, you reference it as 'controller.action', and when you call it, it gives back an object that acts as a Rack endpoint you can interact with as expected.

00:10:05.520 This arrangement enables you to integrate Rails applications seamlessly into other contexts. Middleware is notably an excellent example of how Rails can coexist and collaborate with other frameworks.

00:10:24.480 Rack ships with several middleware components, and Rails functions well alongside them. Not only that, but Rails has contributed to this ecosystem, sending various components upstream, such as Rack Lock, which determines whether to apply a mutex around the entire application.

00:10:44.640 This Rack Lock component is now accessible as middleware for use across various frameworks. There are also additional middleware components, like Rack Bug, which acts as a debugging toolbar per request, and which Rails can leverage as needed.

00:11:05.840 Networking services that manage exceptions, like Hoptoad, could also be implemented as Middleware, functioning without depending on Rails explicitly. Additionally, Rails ships with a variety of Middleware like session Middleware that can also be utilized independently in any framework desiring to implement them.

00:11:32.160 We're committed to ensuring that when you require action controller or action dispatch session middleware, you can do so without the risk of pulling in thousands of extra files you may not need. We're actively making sure that expected files are usable downstream by micro-frameworks without concerns about potential memory issues.

00:11:55.680 Now, you might not want to depend on Rails as an application, but realistically, depending on ActionController becomes feasible provided it does not necessitate the influx of various unrelated components.

00:12:13.680 We obviously have to collaborate with developers downstream to ensure that they feel secure relying on ActionController when they only need basic middleware, but that is more of a social and political issue to resolve.”},{

00:12:33.120 In Rails, you can also employ middleware similarly to the way they are used in Rails themselves—effectively through configuration. I've labeled this aspect as an application-level concern because I foresee developers creating plugins that are expected to be set up using configuration methods.

00:12:52.520 Ultimately, the benefit of using Rack extends to integration tests. Instead of Rails needing to set up a multitude of components that are aware of how Rails operates internally, we can simply treat Rails as a black box.

00:13:16.680 In turn, integration tests can act as mock servers. This evolution means testing is not bound to specific magical testing mechanisms requiring specialized APIs; tests result in standard rack responses that developers understand.

00:13:35.520 There is already a component called RackTest that works seamlessly with Rails 3 because it utilizes the Rack standard. We anticipate similar lines of development in Rails 3, which will ultimately enable the Rails integration test framework to also test Sinatra.

00:13:52.800 Now, what's the future of this endeavor? We anticipate numerous interesting possible implementations. For instance, making any element of Rails, Merb, or Sinatra not special means any routing logic could be integrated without concerns.

00:14:08.640 Instead of the Rails router being solely focused on Rails, it becomes Rack-centric. From a user perspective, this means the API remains unchanged, while power users will gain access to new APIs that facilitate the projection of requests into diverse Rack applications.

00:14:25.680 The second category of improvement focuses on enhancing how Rails operates with existing libraries—most importantly, ensuring RM (Object-Relational Mapping) agnosticism. We aim to guarantee that developers can seamlessly use any ORM alongside Rails.

00:14:45.840 When we refer to RM agnosticism, many Rails users often query: 'Why is your ORM agnosticism unique? Couldn't you just use DataMapper as a Ruby library? Rails supports Ruby libraries, so why not simply require the DataMapper gem and use it?'

00:15:01.040 And while that's true—you can absolutely use DataMapper in Rails without issues—Rails has elements that assume you are using ActiveRecord. For instance, numeric helper methods may be optimized for ActiveRecord functionalities, meaning that special behavior occurs when those objects are presented.

00:15:22.400 Over the past two decades, Rails has incorporated several helpful features that align with ActiveRecord, which may not be universal for other ORMs. This 'magic' is beneficial for those using ActiveRecord.

00:15:38.960 In the future with MB 1.0, we claimed to support ORM agnosticism. However, we originally also hardcoded known interfaces for ORMs into the framework. This provided a challenge because it effectively limited flexibility.

00:15:55.280 A more viable solution now is to propose a method that indicates adherence to one of those ORM interfaces, which will allow effective integration without falling back to the old hardcoded methods. This way, developers can still design efficient systems while maintaining sufficient flexibility.

00:16:15.240 To support ORM writers for those attempting to integrate apps with Rails, we need to enhance those integrations without impacting existing Rails users. There are two alternatives for developers wanting to integrate ORMs that haven't been supported traditionally. The first is to ensure compliance with ActiveRecord interfaces. This means ORMs should follow their structure rigorously.

00:16:32.240 ORM writers can also create a shim, which could introduce required methods. For instance, an object might not respond to 'valid?' but via shimming you can bestow this functionality. The second primary approach entails creating a proxy layer that ensures that any active record style ORM behaves similarly to traditional ActiveRecord objects.

00:17:05.360 On top of that, we clarified we would not require dramatic changes throughout Rails to accomplish integration, which means a minimal adjustment could be placed into the code that governs the performance of errors and reviewing object properties.

00:17:35.920 When working on presentations earlier, I mentioned 'error messages for', which utilizes ActiveRecord heavily. The implementation gathers the object you want to render and moves forward from there. Utilizing the new framework with Rails 3 will simplify this—to work effectively, we can simply state 'object.map(&:to_model)' in order to handle object responses.

00:18:00.640 From that point onward, our objective is to improve the development experience while maintaining Rails' long-standing usability for developers. We're conscious of not breaking existing functionality while introducing enough flexibility for developers.

00:18:18.160 Javascript is another area where we're making improvements to Rails. Rails 2.3 managed JavaScript in a non-agnostic way. For example, when you wrote a remote form, the JavaScript code emitted was heavily reliant on Prototype, leading to performance and usability issues if developers opted to shift to a different library.

00:18:34.720 Pending enhancement efforts signify how we can utilize a more agnostic layout for Javascript code in Rails 3—wherein the markup will be emitted instead. The JavaScript libraries will then manage this emitted markup in their respective ways. Ultimately, we make the event handling simpler by enhancing how JavaScript libraries handle events—eliminating great portions of cumbersome repetitive code.

00:19:02.320 This fundamental shift resolves issues that previously plagued developers working with multiple frameworks while using Rails and it ultimately simplifies the use of Rails within their own integrated designs based on the frameworks they prefer. The last area of focus is refining how Rails can work well within third-party libraries.

00:19:36.880 Leveraging modular design principles in Rails 3 allows developers to access Rails functionality seamlessly. For example, by defining clear interfaces for commonly used elements like rendering, we can further streamline integration.

00:20:00.000 By specifying behaviors in Rails, we aim to demystify its constructs while granting developers ample room to implement features smoothly. Perhaps one of the changes includes enabling controllers or even mailers to share similar reusable code to enhance maintainability across the framework.

00:20:28.560 Finally, Rails will serve as a better foundation for frameworks such as Sinatra, by offering utilities built into Rails to easily embed functionalities directly accessible to Sinatra users.

00:20:44.440 In short, we want to create a system that enables developers to leverage Rails functionalities with unparalleled ease.

00:21:00.080 With that being said, there is still a lot more to be done, and I look forward to working together as a community to innovate.

00:21:12.840 Are there any questions? I have time for two questions.

00:21:27.280 In response to the first question regarding whether Rails will include support for an application model to eliminate direct dependencies on ActiveRecord, I advocate for subclassing and suggest pushing forward with a clearer implementation of coding constructs to enhance usability.

00:22:13.360 Another question concerns the issue when using forms, particularly how couch DB interacts with hashes. I don’t anticipate excessive complexity when coding around this aspect and believe applicable measures will be put in place.

00:22:32.960 Thank you all for your questions, and I'm glad to be here discussing these developments with you!