Micro Talk: Organizing and Packaging Rich Javascript Apps with Ruby

More and more developers are facing the challenge of organizing and deploying complex client-side Javascript apps. It turns out there are some excellent solutions to this problem bubbling up in the Ruby ecosystem. I am responsible for two complex Javascript applications at Yapp, and in this micro-talk, I will share a solid solution to this problem using open source Ruby projects.

GoRuCo 2012

00:00:16.560 Um, this is me. I live two blocks from here with my two kids. I have a company called Yapp where I write a lot of Ruby and JavaScript. When I'm not doing those things, I like to play beach volleyball. Our JavaScript apps are getting bigger.

00:00:29.519 There was a time when having a cowboy approach to managing these applications seemed like a good idea. Here's a little New York history: a few blocks over on 10th Avenue, there used to be a freight train running at street level in the mid-1800s. Unfortunately, many people got killed by that freight train, so they decided to do something about it. The solution was to hire the West Side Cowboys, who would ride their horses in front of the freight trains, waving red flags to alert people of their approach. At the time, it was a practical solution, but now I think we need to reassess how we're organizing and packaging our JavaScript applications.

00:01:03.680 As our JavaScript applications grow, we need to apply software engineering principles that we have learned in other environments. This shift is occurring with MVC frameworks and testing frameworks, which help us organize our code in more sensible ways, making it easier to find later. Today, I want to focus on the build system for JavaScript. This build system should support iterative development as well as the deployment process, allowing us to efficiently deliver JavaScript assets to their intended destinations, whether that be the browser or native wrappers.

00:01:27.200 In the real world, JavaScript build systems tackle several key problems: concatenation, which combines everything into one HTTP payload; minification, which compresses files to reduce their size; authoring in another language like CoffeeScript; and deferred parsing and template compilation. The tools we employ are fairly common and likely well-known to many of you.

00:02:02.960 When designing a build system, the approach is straightforward: it's a pipeline. We start with input at one end, and a series of filters transform that input until we reach output at the other end. For JavaScript, this process often involves taking a collection of JavaScript files, wrapping each file in a closure, minifying them, concatenating into a single file, and placing the result into an output directory.

00:02:30.080 There are several Rails-focused solutions available, the most well-known of which ships with Rails 3—the asset pipeline. Outside the Ruby community, developers use a diverse array of tools for similar purposes, including custom-built and generic tools such as Make. With all these options available, you might wonder why I am here talking about Rake pipeline today. There are two main reasons: first, it has pragmatically solved problems for us in my codebase; and second, this library exemplifies what I love about Ruby.

00:03:07.360 A little history for context: in October 2011, a company named LivingSocial needed tooling for an Ember.js app they were developing. They hired Yehuda Katz, who subsequently created Rake Pipeline. About a month later, at Yehuda's request, it was released as open source. The concept was to model this as a build system, allowing us to grow and compose it according to our needs.

00:03:43.680 If you're going to build a build system in Ruby, the first stop is likely Rake. However, I won't delve deeply into Rake because Jim Wyrick, who is here, will cover that better than I could ever explain. Nevertheless, one important aspect of Rake that you should understand is the file wrapper. This allows you to create files, and if the file you create is newer than its input files, the task is skipped. This mechanism enables incremental builds, which is crucial for achieving fast build times.

00:04:16.960 The foundational innovation of Rake Pipeline is that we can dynamically create file tasks from the filters that perform these transformations, capturing the benefits of this architecture with incremental build speed while leveraging Rake under the hood. This allows us to effectively manage our filter pipeline.

00:05:00.720 So what exactly is a filter in Rake Pipeline? It's a subclass of Rake Pipeline Filter that requires you to provide a 'generate_output' method, taking in input collections and generating output as a result. The file wrapper classes simplify the task of knowing where these files are located on disk, offering user-friendly methods for reading and writing them. This design not only makes it simple to write filters, but also encourages sharing them with others.

00:05:48.319 The next key component of Rake Pipeline is its DSL, which allows you to describe how to compose these filters. For example, if we want to use Rake Pipeline, we start by building the pipeline and creating an instance of the DSL class. We specify our inputs, usually JavaScript files ending in .js, and define the filters to be applied to them. Our first filter can be a closure filter, which wraps our input in an anonymous function to provide information hiding and keep the global scope clean.

00:06:28.320 Next, we can minify each file using a filter from the Rake Pipeline Web Filters project, such as the Uglify filter. The output will then be concatenated into a single file, say application.js, which is placed in the dist directory. Thus, we can express our transformation pipeline simply and intuitively.

00:07:11.040 The Rake Pipeline setup also allows for creating a module for shortcuts. For instance, we can define a method that calls the closure filter, then include this module in the DSL. This makes our transformation process even shorter and more readable.

00:07:55.680 The Rake Pipeline and its associated web filters project already provide helpers for minifying and concatenating files, simplifying the coding process further. I challenge anyone to express this transformation process as succinctly in any other programming language as we can do in Ruby.

00:08:39.360 Another significant aspect of the DSL is matchers, which allow you to restrict filters to particular types of files. For example, if we want to incorporate CoffeeScript into our project, we can specify that. The matcher will generate the input for the next step; in this case, the CoffeeScript files will produce JavaScript output. This way, we can seamlessly add another language to our stack with just a few lines of code.

00:09:30.399 Executing Rake Pipeline is straightforward, thanks to the executable provided with the gem. The 'rake p' command builds and generates the output as expected, while 'rake p clean' removes temporary and output files. The 'rake p server' command starts a development server, allowing real-time previews of changes. This server uses middleware that can be integrated into your own projects if needed.

00:10:07.360 In conclusion, I want to thank the event organizers, as well as Matt, who makes writing this type of code possible. Ruby enables us to express ourselves in ways that bring joy to our work. Thank you all.