Talks
Look Before You Import: A Webpack Survival Guide
Summarized using AI
Look Before You Import: A Webpack Survival Guide
by Ross KaffenbergerIn the talk titled "Look Before You Import: A Webpack Survival Guide," Ross Kaffenberger addresses the challenges of transitioning from traditional asset management in Rails applications using the Rails asset pipeline (Sprockets) to a more modern approach using Webpack. He provides insights based on his own experience with this switch and offers practical advice for developers considering similar changes.
Key Points Discussed:
- The Complexity of Webpack: Ross reflects on the initial confusion he faced when first encountering Webpack's configuration, comparing it to understanding an alien life-form. He describes his own journey from Sprockets to Webpack and the lessons learned.
- Differences Between Sprockets and Webpack: He explains the fundamental differences between the two asset bundling systems, particularly in how they manage JavaScript modules. While Sprockets uses concatenation and global scope, Webpack treats each file as a modular piece.
- Legacy Code Challenges: Transitioning from older Rails setups with heavy use of global variables and Sprockets-specific architecture presented significant hurdles. The move to encapsulate dependencies in a module structure required a mindset shift.
- Common Problems and Solutions:
- Directory Structure: Organizing files in accordance with Webpack’s conventions is crucial. Issues arise when developers apply Sprockets principles to Webpack, leading to problems in asset compilation.
- Dependencies Management:** The need to carefully manage dependencies and ensure that libraries like jQuery are properly included to prevent issues with plugin functionality.
- Code Splitting and Caching:** Ross emphasizes the importance of understanding caching and module loading strategies in Webpack to optimize performance and minimize loading times, particularly through asynchronous imports.
- Survival Tips: Ross shares several key takeaways which include:
- Don't expect the same behaviors from Webpack as you would from Sprockets.
- Get to know your dependencies and how they interact in the Webpack ecosystem.
- Consider refactoring legacy code to align with modern JavaScript practices.
- Embrace the flexibility and power of Webpack but remain aware of its complexities.
In conclusion, Ross Kaffenberger encourages developers to approach the transition to Webpack with a positive mindset and to fully understand its inner workings, stressing that while challenging, the benefits of modern web bundling justify the learning curve. He hopes his insights can serve as a guide for those undertaking similar transitions in their own Rails applications.
00:00:10.969
Let's get started. I want you to imagine that you're at your regular Rails team retrospective. Oh wait, imagine that you actually have a regular retrospective, and you're there discussing technical debt, the pain points you have in your development process, performance issues, and the like.
00:00:25.289
Someone on your team chimes in and says, 'I think we should switch to Webpack.' I've read a few tutorials, and I think it should be pretty easy. Now imagine your response. This happened to me, and a year later, I've put together a talk called 'Look Before You Import: A Webpack Survival Guide' for Rails developers.
00:00:38.370
My name is Ross Kaffenberger. I'm at Rasta on Twitter and GitHub, and I want to say congrats to you all. You made it to the end—nearly the end—of RailsConf, second to last talk, so give yourselves a round of applause! When I found out I was speaking before Aaron Patterson, I was both excited and a little relieved. See, I'm excited because he does some amazing work with Ruby, and generally, it's better to speak before Aaron than after him. When I see his profile picture with the flowing locks, I recall that the original title for my talk was 'How to Webpack Without Losing Your Hair.' One of my colleagues reminded me that I might not be the best person to give that talk; it's been a while since my hair migrated to my face. I imagine a session does get carried away from time to time.
00:01:19.720
So when I'm doing amazing things in Ruby, this is what I think I look like. Like I said, my name is Ross, and although this will be a very JavaScript-focused talk, I do love Ruby. I like to think that I've worked hard to make a name for myself in Ruby open-source software. Speaking of great names, there's a Ruby conference coming up next month named RossConf. In my opinion, this is the best-named Ruby conference out there. I'm really honored they named a conference after me! That's not actually true, but it's in Amsterdam next month, so if you're there, let me know how it goes.
00:01:58.020
I work for a company called LearnZillion. We are building curriculum as a service for K-12 education. We're built on Rails and, more recently, Webpack. We're looking for an engineer to join our remote team, so check us out at learnzillion.com/careers. I'm also a dad to a toddler and to a terrier, and I'm often asked by my colleagues what it's like to be a dad and have pets. I can sum up my experience in one word: joy. Every day of mine is filled with tons and tons of joy. Can you imagine carrying around that joy in your hands? If you are inclined to have children or pets, my hope for you is that your days will be filled with joy as well.
00:02:56.480
Speaking of things that bring me joy, let's talk about Webpack. About this time last year, my team had built over several years our Rails app on the Rails asset pipeline. As you can imagine, for an application that's been around for several years, there's a lot of legacy code built up. We had heavy use of global variables in our JavaScript snippets and our Ruby templates, along with tons of Rails asset gems. We weren't quite sure how this would all fit together as we moved forward. Our development experience had started to decline somewhat, as it took a long time for the page to refresh. We were frustrated that our use of source maps was somewhat limited, so we made the decision: let's switch over to Webpack. We had some vague ideas that things would be better, but it turns out for us, the task was not so easy.
00:03:56.840
We had a lot of assumptions about Webpack that came from our experiences with the Rails asset pipeline, which we assumed would just work in Webpack. The problem with having a Sprockets or Rails asset pipeline mindset when using Webpack is a bit like trying to fit a square peg into a round hole. We've made our job harder on ourselves, and now that we've made the switch, we're better informed about how Webpack works and what we would do differently if we were to do it again. This is the talk I would have liked to have seen before I started, and hopefully, regardless of whether you're considering upgrading a legacy app or starting a greenfield project on Webpack, I hope that we can pull out some lessons for everyone.
00:04:45.670
The main structure of this talk will cover three general scenarios that fall under the umbrella of packaging assets for the browser. We'll discuss six problems that our team encountered while making the switch to Webpack, and we'll come away with seven general survival tips that I think will be applicable to anyone considering this change. But first, let's ensure that we're all on the same page about Sprockets and Webpack.
00:05:02.700
They're meant to solve the same problem, which is packaging assets for the browser. By assets, I mean JavaScript, stylesheets, web fonts, and images. It's important to understand that they approach this problem in fundamentally different ways, particularly in how they group files together into one or fewer files that are served to the browser.
00:05:24.489
In Sprockets, when it puts your JavaScript files together, it performs file concatenation. When that JavaScript is evaluated in the browser, it's evaluated in the global scope. Webpack, however, is a static module bundler, meaning each one of your files is converted to a module that has its own scope in the browser. This key difference has significant implications for the types of decisions you can make, and it took us a while to let this difference sink in. We had a vague sense when we made the switch that there would be a lot of improvements.
00:06:00.630
Now, a year later, I'm happy to report that our bet has paid off. We have better development tools within the Webpack environment alongside Rails. I feel that Webpack is a better strategy for managing dependencies. The vast ecosystem bolstered by the large JavaScript community allows us to take advantage of resources more easily. We can write our code in ES6, although I should note that Sprockets 4, which is still in beta, also allows this. Moreover, we can leverage many new features that were not present in Sprockets or the Rails asset pipeline, one of which is dynamic code splitting, which I'll discuss later.
00:06:47.150
Despite all these great ideas, features, and improvements, if you spend some time researching Webpack online, you'll come across quite a bit of emotion regarding it. Some people passionately hate Webpack, often due to the trial and error involved in getting things to work, primarily because we don't always fully understand what's happening. Some express that Webpack is over-engineered and that the effort required to get the configuration right is not worth it to your team. There's even a website devoted to these 'WTF' moments that people have when trying to set up Webpack. Honestly, I've had some of those moments myself, which I'll share today.
00:07:30.010
One key decision point that encouraged us to make the switch was Rails announcing official Webpack support through the Webpacker gem last year. Webpacker basically aims to bring Rails conventions—this 'Rails is omakase' approach of convention over configuration—to Webpack, which traditionally makes no assumptions for you and is fairly unabated. This goes slightly against what we typically understand as Rails' philosophy. This won't be a Webpack tutorial, but I will recommend three resources that helped me understand and get started with Webpack. They are: 'SurviveJS: Webpack' by Juho H. The Learning Academy by Sean Larkin, who is a tireless advocate for Webpack, and 'Webpack from Nothing' by David Copeland.
00:08:48.870
Let's dive into the survival scenarios and the challenges we encountered as we made this switch. We had to address some difficulties with organizing our code, taming the dependencies we were moving over to Webpack, and ensuring predictable caching. Jumping into the first category, one of the first questions we encountered once we got Webpack up and running in a Rails environment was how to organize our directory structure.
00:09:38.540
Taking a step back to the Rails asset pipeline, the default layout for your JavaScript typically involves placing everything in your app/assets/javascripts directory. Some special files are configured for precompilation to be outputted into the public directory when you want to serve these assets in production. You might need to use some Rails configuration to specify which files you want to be included. When integrating Webpacker, a particularly helpful feature is that you can run Sprockets and Webpack side by side. So, what we did was move files over one by one to get things working.
00:10:30.440
You'll notice that there is a special directory under JavaScript called 'packs.' This term was created by Webpacker, but generally, in Webpack, these correspond to the 'entry' points or those bundles that will be outputted into the corresponding public directory. According to Webpack terminology, this is called the output. Importantly, no extra configuration is needed; anything that goes into the packs directory—specifically JavaScript files—will be precompiled into the public directory. Unfortunately, a lot of folks get this wrong. I've paid a lot of attention to Webpacker issues, and many users report problems that stem from misunderstanding how to organize their directory structure.
00:11:27.920
Instead of structuring their files as outlined, many users mistakenly mimic the layout they were accustomed to in Sprockets, leading to issues. Essentially, they load everything into the packs directory. This is a common mistake because it feels like a natural transition from what they used to do in the Rails asset pipeline. However, if you run your Webpack build and pay close attention to the output, it will reveal what’s happening during the build process. You'll see every individual module from the packs directory getting precompiled into the public directory.
00:12:23.780
This is likely not the desired behavior. The main takeaway here is to only place JavaScript files that you want to serve as entry points in the 'packs' directory. Consequently, you should observe a limited number of files in the output. Survival tip number one: do not assume that just because things functioned in a specific way in Sprockets and the Rails asset pipeline, they will operate the same way in Webpack and Webpacker.
00:13:12.680
Another issue that arose was how to replicate 'require_tree' functionality in Webpack. We enjoy this fantastic directive in Sprockets that allows us to include all the files in a directory automatically with just one line. However, this doesn’t work by default in Webpack. If you try to import a directory in your ES6 code, additional work is required to achieve similar functionality. A Webpack-specific API called 'require.context' functions similarly to a directory glob in Ruby.
00:13:57.880
You would use this to construct a list of files within that directory if you had an index directory or file that essentially serves as an alias to that directory name. From there, you have to iterate over those files and require them one by one. If you ever need to pull a component from that directory, you’ll need to use the 'require.context' method to aggregate that group of modules. You need to give them a name—most commonly, this will be based off the filename—then export that from the index file so it’s accessible to other modules in your system.
00:14:46.810
The takeaway is that while you can achieve much of what Sprockets allows, you must forfeit some convenience, similar to what you'd find on the Ruby side. Webpack offers great flexibility and control, but the APIs may not always be as user-friendly as you might hope. It's easy to feel overwhelmed when delving into these low-level concepts. Moving on to taming dependencies: one strategy from our Rails asset pipeline we wanted to extend to Webpack is code splitting. This allowed us to create an 'application.js' and 'vendor.js' that we included on the page at the same time.
00:15:25.320
The strategy behind this was to enhance our long-term caching. We would place large libraries, such as jQuery and Lodash, in our 'vendor.js' bundle while placing the remainder of our source code in 'application.js,' which could include some vendor libraries such as various jQuery plugins. However, when we attempted to transition this concept of manual code splitting over to Webpack, a problem arose.
00:16:15.860
We aimed to create an 'application.js' and 'vendor.js' pack in Webpacker, and instead of include tags, we started using pack tags. Once we imported jQuery from our Node modules and incorporated it into our bundles, we encountered issues. It's important to note that we needed to ensure jQuery remained available in the global scope due to the code in our Ruby templates relying on this. We could instruct Webpack to make this adjustment just for jQuery.
00:16:51.280
Unfortunately, this had serious consequences on what we observed. We added jQuery and made it available in the global scope through our configuration, then included the chosen.js plugin. This plugin is quite mature and hasn't been updated in a few years, and we tested in the dev tools in the browser, confirming that the chosen function was added to the jQuery object in the global scope.
00:17:34.140
Upon integrating the slick carousel, another jQuery plugin, which was more active and better maintained, we noticed something unexpected. The chosen function suddenly stopped working, which perplexed me at the time. I was shocked to find that the same approach I had used in Sprockets wasn’t effective in Webpack.
00:18:00.990
Fortunately, there is a helpful tool you can add to your development environment to elucidate what’s happening in your Webpack build: the Webpack Bundle Analyzer. Webpacker provides a configuration to help facilitate the addition of this tool, and the Webpack Bundle Analyzer visualizes the size and contents of all the modules and bundles you are building. You can click through the visual to get various statistics.
00:18:50.360
After adding these plugins and jQuery, our analysis revealed a significant issue. On the left, you see our 'vendor' bundle, where jQuery is correctly included. However, the 'application' bundle unexpectedly also contained jQuery, where it should not. This essentially resulted in a conflict because only one version of jQuery can exist in the global scope. This meant any plugin we had added at some point got overridden. A look at the source code of these plugins clarified the issue. The chosen.js plugin assumes jQuery is in the global scope, while slick carousel operates differently.
00:19:45.290
Slick carousel doesn't make that assumption; it checks for the presence of jQuery and adds it as a dependency if necessary. This caused problems, as we could not have multiple instances of jQuery in the global scope. To resolve these issues, we needed to adjust our setup and ensure common imports were handled together. Instead of doing manual code splitting, we allowed Webpack to manage this for us, using the CommonsChunkPlugin, which extracts all common modules into the vendor bundle.
00:20:38.120
I wanted to highlight that this won't work seamlessly in Webpack 4, so it's better to grasp the main concepts involved. This plugin will ensure jQuery is only in the vendor bundle and will avoid duplication later on. As a result of these changes, we reconfirmed our bundle analysis and saw jQuery was now rightly included in the vendor bundles, while our application bundles became smaller since common modules transferred over to the application bundle, appearing once instead of twice.
00:21:32.680
This experience leads me to my third survival tip: you must thoroughly understand your dependencies in Webpack. They might unexpectedly introduce complexities to your configuration. Another challenge we encountered is what I refer to as module shimming, which involves applying transformations to your dependencies. We already manage similar transformations in Sprockets through preprocessors that convert CoffeeScript or ES6 code into standard JavaScript. Webpack has more powerful tools for this purpose, but it refers to them as loaders.
00:22:15.260
In our case, we used an exposed loader to place jQuery in the global scope. We realized we needed to adjust how we worked with an old dependency: chosen.js. By leveraging the imports loader, we imported the required dependencies directly, allowing us to avoid relying on globals. We started moving away from the Sprockets paradigm of using global lookups.
00:23:02.020
Survival tip number four: if you have legacy dependencies, I highly recommend updating them with loaders to ensure compatibility with Webpack, or eliminating them altogether if feasible. We understood that Webpack encourages you to adopt modern practices and paradigms. Now, let's discuss predictable caching, specifically concerning fingerprints. When we ran our build, the two bundles, as done in the Rails asset pipeline, received fingerprints appended to filenames, offering a digest of their contents.
00:23:39.730
We strive for these digests to be consistent unless contents change, allowing us to maintain longer caching periods. If contents alter, we expect the digest to reflect this. For instance, in our application pack, importing a module should ideally not change the vendor bundle's fingerprint unless necessary. However, to our dismay, we found the fingerprints of both bundles changing undesirably, leaving me bewildered, prompting me to deeply reassess how Webpack functions. When you compile your Webpack bundles, it starts at the entry file and walks through all import statements to collect dependencies, leading to a directed graph of interlinked dependencies.
00:24:45.720
Each module within this setup receives a unique ID as an index into an internal array. The first entry file receives ID 0, and so forth. Simultaneously, as we've discussed, we manage two primary bundles: vendor and application.js. As we pull common modules into vendor.js, any adjustments made in application.js necessitate updates to IDs across the board, impacting vendor.js as well due to the shared runtime we established. Consequently, even minor changes could inadvertently affect the vendor bundle’s fingerprint.
00:25:55.020
To address this issue, we utilized another instance of the CommonsChunkPlugin to separate the runtime from the vendor bundle, as well as implemented a hashed module IDs plugin to represent module content in their identifiers rather than numerical IDs. With these improvements, the runtime became a standalone entity; if application.js changed, it wouldn't affect vendor.js, allowing us to keep vendor caching intact. While this compromise was necessary to enhance vendor caching, it offered a clearer path forward.
00:26:54.310
Survival tip number five: take time to learn how the Webpack runtime operates. Understanding these components is essential for effective utilization of Webpack within applications. For my final point, I want to highlight a significant advantage of migrating to Webpack—the ability to handle asynchronous bundles. For instance, at LearnZillion, we utilize PDF.js to render PDF documents into canvas elements, but only on a specific page.
00:27:20.400
Loading this library on every page would be unnecessary. We can rewrite the import from a statement to a function call. This alteration changes the behavior of Webpack; this method defers the loading of that package until the function is evaluated in the browser. Consequently, Webpack generates a separate bundle for this dependency, requesting it on demand—only when that particular line gets evaluated.
00:28:20.040
This mechanism allows for greater efficiency, only fetching what's required on demand. If you use this type of import, don't forget to employ a magic comment within the function call to ensure the chunk receives a consistent name. Running this build confirms the creation of an additional bundle, utilized on an occasional basis, significantly reducing the size of initial bundles loaded on page access.
00:29:11.450
Survival tip six: prioritize leveraging asynchronous bundles when using Webpack to maximize its benefits. We’ve covered many issues, and though you might not encounter the same challenges we've faced, I hope you can appreciate the broader perspective of adjusting the mental model transitioning from Sprockets to Webpack.
00:30:00.480
As the author of a relevant blog post noted, Rails with Webpack isn't for everyone. Don't switch just because it’s considered trendy. Assess the trade-offs: if you have a JavaScript-heavy app and want to exploit modern tools and features like asynchronous bundle loading, it may be the right choice. It will likely take time to learn how this works, while others may find a solution like Sprockets 4 sufficient. The Webpacker framework still needs additional refinement and support.
00:31:09.750
As referenced in DHH's opening keynote, it's a leaky abstraction. Though it conceals complexities you’d otherwise manage setting up Webpack independently, I found that a fundamental understanding is still essential. This is an opportunity for improvement, but to benefit, you must stay current, as Webpack evolves rapidly.
00:31:33.930
So for my final survival tip: recognize and respect the fundamental differences between Webpack and Sprockets. They function in substantially different ways, adopting distinct philosophies on how JavaScript should be bundled. Webpack strives to guide you towards smaller bundles rather than larger ones and encourages you to minimize the use of global scope code and jQuery. These considerations may force a reassessment as you decide to make the switch.
00:32:34.430
Most of all, I recognize that transitioning to Webpack can be daunting for first-time users. Maintain a positive mindset; frustrations will arise. Keep these tips in mind; revisit them when needed, and I assure you, you will survive your journey with Webpack. The slides from this talk will be available on my website, rasta.net/talks. I'm always eager to answer questions about Webpack or other Ruby and Rails subjects, so feel free to reach out. Thank you all for attending RailsConf, and thank you for being part of my talk. Please get home safely after Tenderlove's talk tonight, and I hope to connect with you all soon. Thank you!
Explore all talks recorded at RailsConf 2018
+98