00:00:20.720
Hello, thank you all for being here today. We're going to explore progressive web apps and how they actually fit really well into Rails. I'm going to show you a demo of a progressive web app I created that is very fast and responsive, built entirely with a Rails stack and minimal front-end JavaScript, aside from some Stimulus.
00:00:32.579
My name is John Beatty, and I'm from Loudoun County, Virginia, in the suburbs of Washington, D.C. We live about eight miles from West Virginia, a state described as almost heaven. So, I like to say that we're almost almost heaven. We're also just eight miles from the cloud—literally, Amazon’s US East One region is just down the road, and I even used the local cloud to access the internet.
00:00:48.780
This is my children accessing the Netflix cloud. My Rails developer story starts with developing iOS, Android, and BlackBerry apps back in 2010. I quickly learned that those apps needed a back-end service to manage the data that appears in an app. At this point, I was young and my time was cheap, so I learned Rails to build those backends. I fell in love with Rails because it matched how iOS apps were developed, even down to some of the similarities between Objective-C and Ruby.
00:01:09.570
In the meantime, I worked at a few startups in the D.C. area and also worked on my own projects for a bit. Then, five years ago, an opportunity came up for me to teach and run the IT infrastructure at a school, which took me out of the technology hamster wheel. There, I morphed my job into that of an embedded software engineer, and I've been building a school management system from scratch to manage student records.
00:01:22.140
This role has allowed me to experiment with many Rails features to see where they can be used effectively. I currently blog about Stimulus.js and creating responsive and interactive websites on my blog, JohnBeatty.co. But enough about me! I want to talk about what you are going to get out of this talk.
00:01:34.140
I would like you to walk away from this talk feeling like progressive web app badasses. I want you to be able to confidently discuss progressive web apps without hand-waving your way out of the conversation. There is a lot of confusion surrounding progressive web apps, especially regarding how they can fit into a Rails stack.
00:01:46.229
I want you to be able to advocate for using the full Rails stack when building a fully interactive and fast web app without having to resort to additional JavaScript frameworks on top of a Rails API. I'm going to show you one way I've done this with a version of the Hacker News progressive web app that I built using only Rails.
00:02:00.949
To start, I want to establish a common vocabulary we can use to discuss progressive web apps. This will help ensure that we're on the same page and can cut through the fog surrounding the topic. The most important thing to understand is that a progressive web app is essentially a convention. It’s very simple: you just need to add a couple of files to your app, and a browser will treat it as a progressive web app.
00:02:20.250
All you need is an HTML page, a JSON manifest file linked to from that page, and a JavaScript service worker loaded from that page. There's a bit more to it, but those three components are the essentials. For example, your index page's head would look like this: you would note the link to the manifest file, and the rel tag that specifies it's a manifest.
00:02:38.180
We need to set the viewport so that it appears properly on all screens, especially on compact mobile screens. You can also set a theme color, which would be part of the loading splash screen if someone were to install it as a full-time app. The body of the HTML page is quite simple; it just contains a title, a paragraph, and a bit of JavaScript to load in a service worker from our webpage.
00:02:55.639
We register it with a scope usually set to the full scope of your application. That’s truly all you need for the HTML part of this. The manifest file is similarly straightforward; it’s just a JSON file that contains metadata for the browser, such as the app name and some app icons that will show up when a user installs it.
00:03:16.800
The manifest also sets up the scope of the progressive web app, which you can think of as the root folder from which everything works. This metadata is similar to what you would include in a mobile app, such as the name and icons. Lastly, the service worker needs a few callbacks to work correctly.
00:03:31.960
This JavaScript worker is also a convention, wherein the browser runs a piece of JavaScript code that waits for events to fire and handles them. The first callback is the installation callback, which is used to cache the files you want available offline. At the very least, you should cache a special offline page.
00:03:47.210
You don’t necessarily have to cache anything else if you don’t want to; later, we will use this part to cache our WebPacker assets and any files we think should be available when the web browser is offline. The next callback is the service worker activation callback, which is used for various things, such as managing the state of the app caches and clearing out stale caches that are no longer needed.
00:04:03.270
Finally, we have the service worker fetch callback, which is where the progressive web app really shines. Every single request goes through this function, which decides how to handle that request. First, it tries to retrieve each request over the network. If that fails, it attempts to fetch the file from the cache.
00:04:17.349
If that also fails and it's an HTML request, the worker will load the offline page that we cached earlier. There’s a Mozilla service worker reference page that I’ll link to later in the talk that contains many more examples of what you could achieve within this callback.
00:04:35.120
The example I shared is quite simple, and there are a lot of more powerful things you could do with a service worker. Lastly, we have to register these functions so they actually get called on the relevant events. That’s it! You have three files, and you have a valid progressive web app.
00:04:49.000
Some resources you can explore include Microsoft's PWA builder, a nifty tool that allows you to generate manifest and service worker files for an existing site. You can check some boxes for features you want, and it can be found at PWABuilder.com.
00:05:01.960
Mozilla also has a service worker cookbook with many examples of service worker techniques, including caching content and sending web notifications. I'd highly recommend you check out ServiceWorker.rs. Google Chrome comes with a built-in audit feature that tests for performance, progressive web app setup, accessibility, and best practices on your page.
00:05:15.500
You can find it in the developer tools; just click on audits. You can test performance under mobile networks like 3G or slow 3G connections, which can give you valuable insight.
00:05:31.460
To solidify my claim that this is indeed a progressive web app and that it truly follows a simple convention, I wrote what I call the world's fastest PWA. The source code is available on GitHub if you're interested. But I've run it through Google’s Lighthouse audit.
00:05:50.010
This is an excellent tool for testing these types of applications, and it received a score of 100/100, indicating it's very fast. The app consists of two icon files required for the PWA, an index.html file, a manifest.json file, an offline HTML file that displays when the page is offline, and the service worker. All of that is included in the previous slides.
00:06:05.800
To see it live, you can visit GitHub at johnbeatty.github.io/worlds-fastest-pwa. If you load the page, then turn off your browser or Wi-Fi and refresh, you’ll also get the offline page.
00:06:20.249
So why should we care about this new convention? As I've mentioned, I got into Rails by writing backends for my iOS apps. Anytime I told someone I wrote iOS apps, they would share their great app ideas with me. I think we’ve all had that amazing app idea at some point, convinced it would make us a million bucks.
00:06:35.010
Often, I found myself working on apps that were merely wrappers around some kind of website content. Some of these applications were more complicated and truly needed to be native, but often, they were just pulling RSS feeds and displaying that natively.
00:06:52.130
These weren’t really games, but the native presence helped in some aspects. Imagine being a restaurant; your frequent customers would appreciate the menu loading instantly after they browsed it the first time.
00:07:07.440
Progressive web apps can eliminate the need for many types of these native app wrappers. They provide offline support, and on Android, you can even issue notifications. You can also receive notifications on macOS using Safari, though setting that up is a bit trickier. Most importantly, you have a canvas on which you can build an app that feels and looks like a native app.
00:07:21.199
If we can make an app look and feel like a native app, isn’t it essentially a native app? We’re going to delight our customers by matching the perceived speed of a native app. We’ve already seen some of the simple elements needed to make something a progressive web app, such as the JSON manifest and the service worker.
00:07:38.160
However, to truly delight our customers, we need more than just those three text files. I propose we can use Rails' default components to build fast websites. We can sprinkle Stimulus on the front end for interactivity, use Action Cable for real-time communication between the app and the frontend, utilize Active Job to perform background tasks, and employ Russian Doll caching throughout to reuse back-end HTML rendering each time a page loads.
00:07:55.520
Nothing fancy, just vanilla Rails. I put together a demo based on the Hacker News progressive web app specification, built entirely with Rails. By combining all these components and techniques in the standard ways recommended in various blogs, I was once again able to produce a website that scored 100 on Google’s Lighthouse test.
00:08:11.750
At the same time, it remains a fairly interactive progressive web app. For this Hacker News progressive web app, the data is loaded in the background, and the pages users see on their browsers—whether on their phones or desktops—load incredibly quickly. They'll be updated in the background as the API refreshes.
00:08:29.330
The source code is accessible on GitHub, and while I wasn’t able to clean it up as much as I would like, it’s all available there to give you an idea of how you might structure an app like this. Before I delve into those details, one theme from my time as an iOS developer is always emphasized: ensuring our apps are truly interactive.
00:08:51.470
In the iOS and Android world, every app has a main thread where everything runs. For apps that load data over the network, there’s a rule of thumb: never block the main thread. You want a smooth experience for your users!
00:09:05.830
I believe our web apps can adopt this rule, and we can do it effectively, especially if we’re competing with these mobile apps that serve as wrappers around websites. We have to consider that native apps use the same unreliable cellular networks. If we can provide the illusion of speed by loading something quickly and filling in the data as it becomes available through WebSocket connections, we set ourselves up for success.
00:09:18.770
This works incredibly well with Rails, as I will demonstrate. To reiterate, Turbolinks helps by keeping the JavaScript and CSS parsed, so it doesn’t need to be re-rendered every time a page loads. We’re effectively just waiting for HTML to come over the wire.
00:09:36.640
Any tasks that might slow down the web page and can’t be cached—like accessing another API—can be placed inside Active Job. Once we have the results, we’ll send them over Action Cable to the frontend. The service worker will also aid in this scenario by caching pages that have already been visited. There’s a technique on the Mozilla service workers page that describes how you can load a page from cache and then send a network request to get a new version.
00:09:59.800
When the new page arrives, it can be dynamically loaded, which is similar to what Turbolinks accomplishes. Now, let me present the Hacker News progressive web app built with Rails.
00:10:15.720
The concept of the Hacker News progressive web app emerged while I was researching potential projects. The objective was to build something really interactive, and there were numerous examples created with front-end frameworks like Angular or UJS.
00:10:30.180
I wondered if I could develop something similar using Rails. It seemed like a fitting personal project, particularly since it involved cutting-edge browser technologies. The idea of the Hacker News progressive web app builds upon the to-do MVC specification, which is technically complex but standardized.
00:10:46.620
This allows multiple developers to create their own implementations based on the same specification, demonstrating how different tools solve the same problems. My goal was to prove that Rails could perform just as well as any front-end framework for delivering fast, interactive websites.
00:11:02.480
This became a worthwhile benchmark, and here are the results. You can find it at my website, hn.pwa.johnbeatty.co. It’s utilizing the Bulma framework as a CSS theme and is currently running on Rails 6.
00:11:18.120
It's pulling data from the Hacker News API, and regarding performance, it scores excellently as it implements many recommended Rails techniques. This includes caching, which is served via Passenger through Nginx, with all assets cached and gzipped.
00:11:33.860
The application performs very well. There’s another test required by the Hacker News progressive web app specification—it assesses performance under the demanding conditions of slow 3G connections, often referred to as the emerging markets mobile test.
00:11:51.180
This scenario simulates a very slow 3G connection on a mobile phone, and again, our app performs admirably. The data comes from the Hacker News API, which offers a specification on GitHub. There are five endpoints from which we pull data, accumulating various items across different categories.
00:12:06.320
We also load the details of those items and display them in an engaging format. The Hacker News API consists of simplified models; one is an 'item,' which categorizes everything from jobs and stories to polls and comments.
00:12:23.180
Each item offers all pertinent displayable information, such as titles, any related text, authors, and relationships like parent and child, especially with comments. Then there's a 'user' model, presenting the username of the person who commented and certain biographical details.
00:12:39.500
The data model is quite basic, but I tweaked it slightly to fit well with how we planned to design this Rails app. Each incoming app item is registered as an item, and we categorize those items into five different categorized records.
00:12:56.810
An item can serve as both a ‘new’ item and a ‘top’ item depending on its success, and each narrative must correspond with a category. For example, top stories have their top item and the new stories have their new item, all the way down the line.
00:13:12.660
These records also track the position of the items as they appear, allowing us to preserve the order specified by the API when retrieving data. Moreover, items maintain a parent-child relationship, particularly regarding comments. The story is usually the parent item with numerous comments beneath it.
00:13:29.920
To match this structure, our web pages consist of five distinct feeds that we must line up with specific API endpoints: '/top,' which fetches all top stories; the same sequence applies to new stories, etc. This structuring guarantees our pages load quickly.
00:13:46.470
As the top page hits the top stories endpoint, we receive up to 500 different IDs ordered from 0 to 499. Following this, we create 500 unique requests to the Hacker News API to extract the details of each ID since there's no bulk retrieval option.
00:14:03.090
This limitation justifies our decision to utilize background jobs; if one fails, it can be restarted using something like Sidekiq to manage background jobs. Each item job will independently retrieve details, highlighting the importance of keeping the main thread unblocked.
00:14:20.150
If we were to load all top items directly, we could face a significant wait time just to render the HTML page. Offloading these tasks to the background improves performance and allows us to effectively delight our customers.
00:14:41.320
Our Active Job setup is direct; it is an HTTP GET request to the API, which retrieves JSON data comprising an array of items. The system processes one item at a time, lodging each within corresponding records. Terms are stored and updated, ensuring efficient management.
00:14:58.040
When items are no longer part of the Hacker News API response, we delete them from our records. As for retrieving each item, associated top items collecting pertinent details are established through a 'load top item job,' which retrieves and stores this information effectively.
00:15:14.740
The integration of Action Cable is beneficial here. When users visit the front page, there's no need to re-render all those items; they exist in our cache of HTML items. We repeat this approach for loading new, shown, and active job items, ensuring that performance gains are maximized.
00:15:31.160
The code outlined above conveys how the API loads while providing data as JSON feeds into Active Record models for items and related top items. We transmit these changes over Action Cable to the frontend, dynamically rendering HTML in response.
00:15:46.760
This rendering remains entirely server-side; we don't need any front-end formatting, ensuring aggressive Russian Doll caching can be utilized. Even if theoretically there are many queries in this setup, the caching helps maintain the performance of the page.
00:16:02.060
The top news controller pattern is consistent for all other categories; we obtain a page parameter for pagination. The page starts at 0 and for every page, we generate 32 items. We fetch the top item most recently updated to facilitate caching and load those items.
00:16:18.570
In the show HTML file, we again implement caching on the news list, the top item, and the page to maintain its individual cache, preventing duplicates. The top items are sliced and rendered, with each cached individually.
00:16:34.520
The caching operates on three levels, and since we pre-render many of these partials during API fetches, we achieve solid performance. Aggressively caching content permits rapid display while reducing database queries, giving users a seamless experience.
00:16:50.550
The frequent updates taking place in real time are sent over the WebSocket connection through Action Cable. As items update in the background, they are broadcast to any listeners; different controllers (including Stimulus controllers) manage these Action Cable connections.
00:17:06.660
The Stimulus controller subscribes to both the individual item updates and location updates. The location index for each top item allows clients to remain synced with ongoing changes.
00:17:22.960
Since this relies heavily on background operations, users can view the page in real time while the data refreshes. It creates a strongly interactive experience, with the HTML transmitted over WebSocket connections.
00:17:39.920
As a reminder, the Stimulus controllers manage connections to the Action Cable channels. In previous slides, I featured an item location controller, which leverages data location attributes from each top item.
00:17:55.410
The first section of the item location stimulus controller captures the channel names from the data attributes. It ensures seamless subscription management—whether a page is being refreshed or if a new page is opened without reloading the entire JavaScript code.
00:18:14.320
In the received message section of this channel, previous HTML on the page is cleanly replaced with newly rendered HTML. While fancy animations and transitions could be introduced through CSS, this example focuses on replacing HTML swiftly without them.
00:18:30.770
This represents the second half of the stimulus controller. In the connect phase of the controller’s operation, it attempts to listen for Action Cable connection updates. If a connection setup is successful, it subscribes; if not, it simply won’t function.
00:18:46.960
When the stimulus controller initializes, it establishes the channel. If the page refreshes or navigates to a different page, the controller issues an unfollow command, ceasing any extraneous data streams.
00:19:03.360
In final details, a listen function collects location IDs based on the current page for an effective follow command on the channel. Each location is indexed from 0 throughout the item collection.
00:19:19.320
Now, having crafted an interactive website, we need to convert this into a PWA to fulfill app requirements, and there are many potential methods. The approach I adopted centers around caching all webpack assets for effective delivery.
00:19:36.230
Consequently, I created a dedicated controller to handle routes for the service worker, the manifest file, and the offline file—all leading to this ServiceWorker controller. The controller doesn’t require much functionality, just a stop of forgery protections for the Service Worker, as Chrome requires.
00:19:51.110
Here’s the manifest.json file, leveraging Asset Pipeline, with URLs generated via Rails helpers. Ensuring assets don’t suffer from caching was critical; we want them to be fresh each time a browser requests them. Hence, this structure lets us customize how assets function per request.
00:20:07.350
Consequently, we’ve set routes for the service worker, manifest, and offline files that route through the single controller. The core of the Service Worker code can be streamlined. The essential element would be caching the assets from Webpacker without manual adjustments.
00:20:23.260
To summarize, you can construct an exceptionally rapid web app that rivals any front-end JavaScript solution. By leveraging Turbolinks, your app solely transmits HTML beyond the first load. It’s fundamentally similar to any other web app requiring all resources to load initially.
00:20:40.600
It connects with a JSON API, whether text is HTML or JSON; the performance remains comparable to any front-end framework. The Russian Doll caching means that each request doesn't require constant re-rendering, significantly reducing database calls by reusing partials.
00:20:57.860
Active Job effectively performs massive processing by loading hundreds of items simultaneously and passing that data back to the frontend out-of-band through Action Cable. Both Action Cable and Active Job collaborate to maintain an open perceived main thread.
00:21:14.680
Moreover, Stimulus will add interactivity, allowing anything not requiring server-side requests to function seamlessly on the client side.
00:21:30.810
In conclusion, I hope this insight inspires ideas for enhancing your Rails applications, regardless of whether you aim to build a progressive web app. The tenet of avoiding main thread blocking is valuable regardless of the experience you wish to deliver.
00:21:44.280
Progressive web apps could even replace native Android apps, particularly as they gain first-party support on Android—if not now, then in future releases. The turbulence of the Android wrapper is certainly in flux, which offers opportunities to implement a PWA without worrying about the native Android app.
00:21:58.600
You could even consider using a progressive web app instead of a native iOS client, should the need arise. Thank you very much! You’re now able to cut through the front-end fog and confidently maintain Rails at the forefront while building a progressive web app.